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(57) On a BD-ROM, there is recorded is a digital 
stream into which video and audio streams are multi- 
plexed. The video stream Is composed of a plurality of 
pictures that together represent video. The graphics 
stream includes a PES packet storing state control in- 
formation (ICS) and PES packets containing graphics 
data (ODSs). The graphics data constructs an interac- 
tive display. The state control information defines the 



control so as to change the states of buttons presented 
on the interactive display according to the reproduction 
proceeding and user operatfons. 

The ICS is attached with a PTS showing the display 
timing of a picture to be synchronized with the interactive 
display. The CDS is attached with a PTS showing the 
timing for decoding the graphics data. The timing shown 
in the CDS is prior to the display timing. 
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Description 
TECHNICAL FIELD 

s [0001] Ttie present invention relates to a recording medium such as a BD-ROM for distribution of movies, and aiso 
to a reproduction apparatus for such a recording medium. More particularly, the present invention relates to an im- 
provement in interactive control techniques. 

BACKGROUND ART 

10 

[0002] It lias long been desired to realize such interactive control that buttons appear on a display screen as a video 
stream is reproduced and that the reproduction proceeds in accordance with user operations made to the buttons. A 
DVD is a breakthrough recording medium realizing such reproduction control The synchronous presentation of buttons 
with a video stream is established through the use of time stamps set to cause the buttons to appear at specific points 

15 on the reproduction time axis of the video stream. 

[0003] In order to realize the interactive control, however, it is not sufficient to record onto a recording medium graph- 
ics data used for rendering the buttons. A reproduction apparatus needs to be controlled so as to change the state of 
each button displayed on a screen, according to a user operation or to the proceeding of video stream reproduction. 
To realize the interactive control, a stream into which audio and video streams are multiplexed (Video Objects) is 

20 recorded on a DVD, and a NAVI pack containing state control information is provided at the beginning of each VOBU. 
The VOBU includes one GOP of the video stream, and also includes audio data and graphics data, which is supple- 
mental to the video stream, both to be read simultaneously with the GOP from the DVD. The state control Information 
is used to change, according to a user operation, the state of each button displayed on the screen. The NAVI pack 
contains information defining a transfer rate and a buffer size that each stream requires in handling the GOP. Since 

25 the DVD stores the state control infomiation in NAVI packs, the button states can be changed with the time accuracy 
of GOPs. FIG. 1 illustrates the interactive control described above. In the figure, the lowest level shows data allocation 
on the DVD. It is shown that state control inf onnation is contained in a NAVI pack. The state control information remains 
valid during a time period of the GOP to which the NAVI pack belongs. Each graphics object is contained in a PES 
packet, and displayed at the same timing with a picture to be synchronized with the graphk^ object. This prior art 

30 technique is disclosed for example in JP patent No. 2813245, 

[0004] It should be noted here that in DVD authoring, the structures of GOPs and VOBUs are not determined until 
elementary streams of video, audio, and graphics data are encoded and ready to be multiplexed into one VOB. The 
mult^lexing is a final stage of authoring. That is, the state control information cannot be incorporated into a VOB before 
this final stage, and thus testing of how the buttons change on a display screen can not be carried out any eariier. 

35 Consequently, it is often the case that bugs are found right before the shipment and the developers are forced to make 
corrections in a rush, in addition, without enough time for the testing, it is risky to Incorporate buttons of complex 
animation into a movie. For the reasons stated above, it is common in current authoring to Incorporate relatively simple 
buttons, such as one which changes in color in response to a user operation. 

40 DISCLOSURE OF THE INVENTION 

[0005] The present invention aims to provide a recording medium which allows the button state changes to be tested 
at an early stage of authoring. 

[0006] It has been an effective scheme to store state control information in a NAVI pack of each VOBU when con- 
^5 sidering a DVD's data read rate. This is because the state control information is read from the DVD simply by reading 
the NAVI pack, so that the bandwidth required for reading is low. 

[0007] However, the scheme is not as effective when considering a rate at which a BD-ROM is read. The BD-ROM's 

read rate is much higher than that of DVD' s, so that It Is no longer important to restrict the bandwidth. 

[0008] In view of the advances in recording mediums and to achieve the above aim, a recording medium according 

50 to the present invention has recorded thereon a digital stream into which a video stream and a graphics stream are 
multiplexed. The graphics stream includes graphics data and state control information integrated therein. The graphics 
data is used to compose an interactive display. The state con^ol information Is used to cause the interactive display 
to change to a different state in response to a reproduction proceeding of the video stream and a user operation. 
According to the recording medium of the present invention, the information for causing the button state change is 

55 integrated with the graphics data into the graphics stream. Thus, as soon as the graphics stream is generated, the 
validation test can be carried out to check how the button state changes in accordance with the reproduction proceeding. 
It is no longer necessary to v/ait for the completion of video stream encoding or of stream muitipiexing. Since the 
verification test of button state changes can be carried out at an early stage of the authoring (prior to the streammul- 



2 



EP1 608 165 A1 



tiplexing), the undesirable possibility is reduced that an error is found in the recording medium right before the shipment 
and the developers are forced to rush. Also, since the graphics stream can be subjected to the verification test inde- 
pendently of the other streams, there is provided a better environment for incorporating buttons of complex animation 
into a movie. 

[0009] Here, the state contro! infonriation in each of the plurality of display sets may include an update flag. When 
set to ON, the update flag indicates that the display set is identical to an immediately preceding display set with respect 
to the state control information and the graphics data, except for a button command. When set to OFF, the update flag 
indicates that the display set is identical to an immediately preceding display set with respect to the state control 
information and the graphics data. The button command may be for execution by a reproduction apparatus upon 
activation of an assodatedbutton on the interactive display. 

[0010] With the above state structure, such a title can be produced that the reproduction branches to one out of a 
plurality of reproduction paths in accordance with buEon commands. For example, a title is a quiz game and two display 
sets present a question for a user to answer. The title may be constmcted to branch to a more and more disadvanta- 
geous reproduction path for the user as the user response delays. 

[001 1 ] Here, each of the n buttons may have a number assigned thereto. Each of the n pieces of button information 
may include a number assigned to an associated button and a flag showing whetherthe button is numerically selectable 
with the number. With this structure, discrete numbers can be assigned to the buttons, which is convenient for producing 
such a title as a fist of baseball players. 

[001 2| To be more specific, such button commands are prepared that the reproduction branches to scenes showing 
play of the baseball players. In addition, the player numbers are assigned to the button commands . With this arrange- 
ment, the reproduction path to the scenes of a specific baseball player is taken at a numeric input of the player number. 

BRIEF DESCRIPTION OF DRAWINGS 



25 [0013] 

FIG. 1 Is a view illustrating interactive control on a DVD; 

FIG. 2A is a view showing a usage pattern of a recording medium according to the present invention; 

FIG. 2B Is a view showing keys of a remote controller 400 for receiving user operations to an interactive display; 
30 FIG. 3 is a view showing a structure of a BD-ROM; 

FIG, 4 is a view schematically shows a structure of an AV Clip; 
. FIG. 5 IS a view showing an internal structure of Clip infonriation; 

FIG. 6 Is a view showing an internal structure of PlayList infomfiation; 

FIG. 7 is a view schematically illustrating indirect references by the PlayList infomnation; 
35 FIG. 8A is a view showing a structure of a graphical stream; 

FIG. 88 is a view showing intemal structures of an ICS and an ODS; 

FIG. 9 Is a view showing a logical structure defined by various types of functional segments; 

FIG. 1 0A is a view showing a data structure of an ODS defining a graphics object; 

FIG. 108 is a view showing a data structure of a PDS; 
40 FIG. 1 1 is a view showing a data structure of an Interactive Composition Segment; 

FIG. 12 is a view showing the relation between ODSs and ICS contained within a DS (n); 

FIG, 13 is a view showing a connposite screen at the display timing of an arbitrary picture pt1 ; 

FIG. 14 is a view showing an example of button information setting within an ICS; 

FIG. 15 IS a view illustrating button state transitions of buttons A-O; 
45 FIG. 16 is a view showing examples of graphical reorientations defined by ODSs 11 , 21 , 31 , and 41 ; 

FIG. 1 7 is a view showing examples of graphical representations defined by ODSs 11-19 associated with the button 

A; 

FIG. 18 is a view showing an example of an ICS and ODSs included in a DS; 
FIG. 19 is a view showing the order of ODSs and button-state groups within 'a DS; 
^0 FIG. 20 a view illustrating button state transitions on an interactive display defined by the button-state groups 

shown in FIG. 19; ' ■ 

FIG. 21 is a view showing the order of ODSs within a DS; 

FIG. 22 is a view illustrating the timing of synchronous display defined by ah^IGS; 

FIG. 23 is a view illustrating the DTS and PTS setting in the case where-a plurality of ODSs constitutes an initial 
55 interactive display and a default„se}ected_button_number is valid; 

FIG. 24 is a view illustrating the setting of DTS and PTS in the case where an initial interactive display Is constituted 
by a plurality of ODSs* and a default selected button number is invalid; - ■ - 
FIG. 25 is a view showing an internal structure of a reproduction apparatus; ' 
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FIG. 26 a timing chart of the pipeline processing performed by the reproduction apparatus; 
FJG. 27 a timing chart of the pipeline processing performed by the reproduction apparatus in the case where a 
default selected button is not specified; 

FIG. 28 is a flowchart of the operations of a controller 20 for executing a LinkPL function; 
FIG. 29 is a flowchart of the load processing of functional segments; 

FIG. 30 is a view illustrating how the load processing is perfomned at the time of a skip operation; 
FIG. 31 is a view illustrating how a DS 10 Is loaded to a coded data buffer 13 of the reproduction apparatus; 
FIG. 32 is a view illustrating normal reproduction; 

FIG. 33 is a view illustrating how DSs 1 , 1 0, 20 are loaded when the normal reproduction is perfomned as illustrated 
in FIG. 32; 

FIG. 34 Is a flowchart of the main routine of processing performed by the graphics controller 1 7: 
FIG. 35 is a flowchart of the synchronization control based on time stamps; 
FIG. 36 is a flowchart of the rendering to the graphics plane 8; 

FIG. 37 is a flowchart of the processing for automatic activation of a default selected button; 
FIG. 38 is a flowchart of the processing for animated presentation; 
FIG. 39 is a flowchart of the UO processing; 
FIG. 40 is a flowchart of the current button change processing; 
FIG. 41 is a flowchart of the numeric input processing; 

FIG. 42 is a view showing an Internal structure of the reproduction apparatus capable of producing a click sound; 
FIG. 43 is a view showing a data structure of an ICS for reproducing a click sound; 

FIG. 44A is a view showing the state control information including buttonjnfo (1 ) and buttonjnfo (2); 
FIG. 44B is a view showing the reading process of the ICS including the state control information; 
FIG. 44G is a view showing an example of three buttons (buttons A, B, and C) that are diagonally aligned on an 
interactive display and an example of button state information associated with the buttons; 
FIGs. 45 A and 45B are views illustrating the reproduction control of the click sound data according to the ICs read 
to the preload memory 21 ; 

FIG. 45C is a view showing an example of three buttons (buttons A, B, and C) tiiat are laterally aligned on an 
interactive display and an example of button state infonnation associated with the buttons 
FIG. 46 is a flowchart of the manufacturing process of a BD-ROM according to a third embodiment of the present 
invention; 

FIG. 47 is a view showing an ICS according to one modification of the present invention; and 
FIG. 48 is a view showing an ICS defining a clfok sound for each key of the remote controller 

BEST MODE FOR CARRYING OLiTTHE INVENTION 

35 

(FIRST EMBODIMENT) 

[0014] The following is a description of a recording medium according to a first embodiment of the present invention. 
First, a description is given to usage of the recording medium, which is one form of practicing the present invention. 

40 FIG. 2A shows a usage pattem of the recording medium according to the present invention. In FIGs. 2, a BD-ROM 
100 is a recording medium according to the present invention. The BD-ROM 100 is used for providing movies to a 
home theater system composed of a reproduction apparatus 200, a television 300, and a remote controller 400. The 
remote controller 400 receives user operations instructing to change the state of interactive display, and is closely 
related to the recording medium according to the present invention, FIG. 2B shows keys of the remote controller 400. 

45 As shown In the figure, the remote controller 400 has a Move Up key, a Move Down key, a Move Right key, and a Move 
Left key. Each button displayed on the interactive display has three states: the normal state; the selected state; and 
the activated state. The Move Up, Move Down, Move Right, and Move Left keys are used to receive user operations 
for causing the button state to change in the order of the normal state the selected state -> the activated state, for 
example. When a button has the normal state, the button is simply displayed. When a button has the selected stated, 

50 the button is currently focused as a result of a user operation but not yet activated. When a button has the activated 
state, the button has been activated. When a specific button has the selected state on an interactive display, the Move 
Up key is used to put a button displayed above the currently selected button to the selected state. The Move Down 
key is used to put a button displayed below the currently selected button to the selected state. The Move Right key is 
used to put a button displayed right to the currently selected button to the iselected state. The Move Left key is used 

55 to out a button displayed left to the currently selected button to the selected state. 

[001 5] An Activate key is used to put a butto n hssAnq the selected state to the activated state (to activate the cu rrently 
selected button) . Numeral keys "0" to "9" are used for numeric selection so as to put a button having assigned an 
inputted value to the selected state. A "+1 0** key is used to add the value of 10 to a value having been entered. Note 
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that the '*0** key and the key are both used to enter a two-digit value. Thus, one of the "0" and "+1 0" keys may 
be sufficient Instead of both. 

[0016] This concludes the description of the usage of recording medium according to the present invention. 
[0017] Next, a description is given to production of the recording medium, which is another form of practicing the 
5 present invention. The recording medium can be embodied by making innprovements to an application layer of a 
BD-ROM, FIG. 3 shows an exampie structure of the BD-ROM 1 00. 

[0018] In the figure, the BD-ROM 100 is shown on the fourth level, and a BD-ROM' s track is shown on the third 
level. In the figure, the track is stretched out into a straight line, although the track in practice spirals outwards from 
the center of the BD-ROM. The track is composed of a lead-in area, a volume area, and a lead-out area. The volume 
10 area has a layer model of a physteal layer, a file system layer, and an apptfcation layer. The first level shows, in a 
directory structure, a fomnat of the application layer (application fomiat) of the BD-ROM. As illustrated, the BD-ROM 
has a BDMV directory below a ROOT directory. The BDMV directory contains files, such as XXX.M2TS, XXX.CLPl, 
and YYY.MPLS. The BD-ROM of the present invention can be produced by creating an application format as shown 
in the figure. 

/5 [0019] Now, a description is given to each of the files used in the application format. First, an AV Clip (XXX.M2TS 
file) is described. 

[0020] The AV Clip (XXX.N2TS) is a digital stream compliant with an MPEG-TS (transport stream) format, and ob- 
tained by multiplexing a video stream, one or more audio streams, a presentation gnaphtes stream, and an interactive 
graphics stream. The video stream represents video of a movie. The audio stream represents audio of the movie. The 

20 presentation graphics stream represents subtitles of the movie. The Interactive graphics stream represents dynamic 
control procedures for reproduction of menus. FIG. 4 schematically shows a stmcture of the AV Clip. 
[0021] The AV Clip shown in the figure on the middle level is obtained in the following manner. The video stream 
shown on the upper first level Is composed of a plurality of video frames (picture pj1 , pj2, pj3, ...), and the audio stream 
also shown on the upper first level is composed of a plurality of audio frames. The video and audio streams are sep- 

25 arately converted to PES packets shown on the upper second level. The PES packets are further converted to TS 
packets shown on the upper third level. Similarly, the presentation graphics stream and the interactive graphics stream 
shown on the lower first level are separately converted to PES packets shown on the lower second level, and further 
converted to TS packets shown on the lower third level. These TS packets shown on the upper and lower third levels 
are multiplexed to form the AV Clip. 

50 [0022] The AV Clip generated as above is divided into a plurality of extents in the same way as computer files, and 
recorded onto the BD-ROM. The AV Clip is composed of one or more Access Units, and the reproduction of AV Clip 
may skip to a point con-esponding to ant Access Unit. An Access Unit Is composed of one GOP (Group Of Pictures) 
and an audio frame to be read simultaneously with the GOP, and is a minimum unit for decoding. GOPs include three 
types of pictures: Bi direct ion ally Predictive picture (B picture) that is coded using correlation with both past and future 

35 pictures; Predictive picture (P picture) that is coded using correlation with past pictures; and Intra picture (I picture) 
that is coded using its own spatial frequency characteristic without referencing the Inter-frame correlation. 
[0023] Clip information (XXX.CLPl) Is management information of each AV Clip. Fig. 5 shows an internal structure 
of Clip information. Since an AV Clip is obtained by multiplexing a video stream and an audio stream and accessible 
at random in the units calledAccess Units, the Clip information includes information, such as the attributes of video 

40 and audio streams and the points within the AV Clip to whfch a skip operation may be perfomried. in the figure, the 
doted lines indicate that the structure of Clip information is excerpted to be shown In detail. As indicated by the doted 
lines hnl , the Clip infomnation (XXX.CLPl) is composed of "attribute information" of the video and audio streams, and 
"EP_map" that is a reference table for searching Access Units. 

[0024] As doted fines hn2 indicate, the attribute information is composed of an attribute of the video stream (Video 
45 Attribute), the number of attributes (Number), and attributes (Audio Attribute #1 -#m) of all the audio streams multiplexed 
in the AV Clip. As doted lines hnS indicate, the video attribute is composed of. information showing the coding method 
in which the video stream is coded (Coding), the resolution of pictures constituting the video stream (Resolution), the 
aspect ratio (Aspect), and the frame rate (Framerate). i 

[0025] As doted lines hn4 indicate, each of the attributes of audio streams .(Audio Attribute #1 -#m) is composed of 
50 information showing the coding method in which a corresponding audio stream is coded (Coding), the channel number 
of the audio stream (Ch.), the language of the audio stream (Lang.), and the sampling frequency. 
[0026] The EP_map is a reference table for indirectly referencing by time the addresses of accessible points in skip 
operations. As doted lines hn5 indicate, the EP_map is composed of a plurality of entries (Access Unit #1 entry, Access 
Unit #2 entry, Access Unit #3 entry...), and the number of entries (Number): As. doted lines hn6 indicate, each entry 
55 shows the reproduction start time and the address of a corresponding Access Unit (Note that the size of a first I picture 
in the Access Unit (l-size) maybe additionally shown) . The reproduction ^tart time is shown by a time stamp for a 
picture located at the beginning of the Access Unit (Presentation Time Stamp). The address is shown by a serial number 
of a corresponding TS packet (SPN: Source Packet Number). Due to the variable-length coding, the Access Units 
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each containing a GOP are not uniform both in size and reproduction time period. Yet, with reference to the entries 
corresponding to the Access Units, an Access Unit located at a point corresponding to any given reproduction time 
can be searched, so that the reproduction can be started from the first picture in the searched Access Unit. Note that 
"XXX" in the file name "XXX.CLPI** is the same as the name of AV Clip to which the Clip information corresponds. !n 

5 the figure, the nie name of the AV Clip is "XXX", which means that the Clip infonnation (XXX.CLPI) corresponds to the 
AV Clip (XXX M2TS). This concludes the description of Clip infonnation. Next, PiayUst infomnation will be described. 
[0027] YYY.MPLS (PlayList infonnation) is a table serving as a PlayUst defining reproduction paths, and is composed 
of a plurality of pieces of Playltem information (Playltem Information #1, #2, #3,... #n) and the number of Playltem 
information pieces (Number). FIG. 6 shows an Internal structure of the PlayList infonnation. PlayList information shows 

'fo one or more logical segments for reproduction each defined by Playltem Infonnation. As the doted lines hs1 indicate, 
the structure of Playltem information is excerpted to be shown in detail. As shown in the figure, the Piayltem information 
is composed of: "c!ipJnformation_file_name" showing the file name of a reproduction segment within the AV Clip to 
which the in_.time and out_time of the reproduction segment belong; ''clip_codecJdent{f{er*' showing the coding method 
by which the AV Clip is coded; ''in_time*' showing the time corresponding to the start point of the reproduction segment; 

'5 and '*out_time" showing the time corresponding to the end point of the reproduction segment. 

[0028] One feature of the Playltem information lies in its notational conventions. That is, a reproduction segment is 
defined by the indirect reference by time, using the EP_.map as a reference table. FIG. 7 is a view schematically 
illustrating the indirect reference by time. In the figure, the AV Clqa is composed of a plurality of Access Units. The 
EP„map In the Clip infonnation specifies the sector address of each Access Unit as indicated by arrows ay1 , ay2, ayS, 

20 anday4. Each of the arrows jy1 , jy2, jy3, and J y4 is a schematic representation of an indirect reference to an Access 
Unit. In short, each piece of Playltem Information has a reference (arrows jy1 , jy2, jyS, and jy4) specifying by time via 
the EP_map, the address of a corresponding Access Unit contained in the AV Clip. 

[0029] A reproduction segment on the BD-ROM specified by a set of Playltem information - Clip information - AV 
Clip is called "Playltem". A logical unit of reproduction on the BD-ROM specified by a set of PL infomiation - Clip 
25 information - AV Clip is called "PlayList (hereinafter, PL)". A movie recorded on the BD-ROM is segmented into the 
logical units of PLs. Since the movie is segmented into the logical units, it is possible to define such a PL specifying 
scenes in all of which a specific character makes an appearance. In this way, separately from the main movie, another 
movie is readily made in which the character appears at ail times, 

[0030] Since movies recorded on BD-ROM have the logical structure as described above, an AV Clip of a specific 
30 scene in a movie may be readily "recycled" or used in another movie. 

[0031] Next, a description is given to an Interactive graphical stream. FIG. 8A is a view showing a structure of the 

graphical stream. On the first level, a string of TS packets constituting an AV Clip is shown. On the second level, a 
string of PES packets constituting a graphics stream is shown. The PES packets shown on the second level are formed 
by concatenating payloads of TS packets having a predetermined PID within the TS packet string shown on the first 
35 level. Note that no description is given to a presentation graphics stream because it is not the gist of the present 
Invention. 

[0032] On the third level, a structure of the graphics stream is shown. The graphics stream is composed of functional 
segments that include an ICS (Interactive Composition Segment), a PDS (Palette Definition Segment), an CDS (Object 
Definition Segment), and an END (End of Display Set Segment). Of these functional segments, the ICS is a screen 

40 composition segment, whereas the PDS, ODS, and END are definition segments. Each functional segment is either 
in one-to-one or one-to-multiple correspondence with PES packets. That is, one functional segment Is recorded on the 
BD-ROM 1 00 after being converted to a single PES packet, or fragmented and converted to a plurality of PES packets. 
[0033] FIG. 88 is a view showing the PES packets obtained by converting functional segment (s). As shown in FIG. 
8B, each PES packet is composed of a packet header and a payload. The payload is an entity of a functional segment, 

45 and the packet header contains a DTS and PTS associated with that functional segment Hereinafter, the DTS and 
PDS contained in the header of a PES packet containing a functional segment are referred to as the DTS and PTS of 
that functional segment 

[0034] These various types of functional segments define a logfcal structure as shown In FIG. 9. In the figure, the 
functional segments are shown on the third level, display sets are shown on.the second level, and Epochs are shown 

50 on the first level. • - 

[0035] Each display set (hereinafter "DS") shown on the second level is a group of functional segments which together 
constitute one complete screen of graphics. Dashed lines hk2 indicate a OS to which the functional segments on the 
third level belong. As can be seen from the figure, the series of functional segments, ICS-PDS-ODS-END constitutes 
one DS. Upon reading these functional segments constituting the DS from the-BD-ROM, the reproduction apparatus 

55 can produce one screen of graphics. 

[0036] An Epoch shown on the first level refers to a time period during which the continuity in memory management 
must be maintained on the reproduction time axis of AV Clip, or to a set of data allocated to the time period. Memory 
mentioned here includes a graphics plane for storing one screen of graphics and an object buffer for storing uncom- 
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pressed graphics data. The continuous memory management means that throughout the Epoch neither the graphics 
plane nor the object buffer is flushed, and erasing and rendering of graphics are performed only within a predetermined 
rectangular area of the graphics plane (to flush means to clear the entire graphics plane and the entire object buffer). 
The size and position of this rectangular area are fixed throughout the Epoch. So long as erasing and rendering of 
5 graphics are performed within this fixed rectangular area of the graphics plane, seamless reproduction is ensured. In 
other words, the Epoch is a time unit, on the reproduction time axis of the AV Clip, during which seamless reproduction 
is ensured. To change the graphics rendering area In the graphics plane, it Is necessary to define a point of change 
on the reproduction time axis and set a new Epoch from the point onward. In this case, a boundary between the two 
Epochs is not seamless. 

10 [0037] The seamless reproduction used herein means that the erasing/rendering of graphics is completed with a 
predetermined number of video frames, tn the case of interactive graphics stream, the number of video frames is four 

to five. The number of video frames is detemnined based on the ratio of the fixed region to the entire graphics plane 
and the transfer rate between the object buffer and the graphics plane, 

[0038] In the figure, dashed lines hkl and hk2 indicate an Epoch to which the functional segments shown on the 
15 second level belong. It is shown that a series of DSs, which are an Epoch Start DS, an Acquisition Point DS, and a 
Nomial Case DS, constitutes one Epoch on shown on the first level. Here, "Epoch Start", "Acquisition Potnr, and 
"Normal Case" are types of DSs. Although the Acquisition Point DS precedes the Normal Case DS in FIG. 9, they may 
be arranged in the reverse order, 

[0039] The "Epoch Start" DS provides a display effect "new display", and Indicates a start of a new Epoch. The Epoch 
20 Start DS thus contains all functional segments needed for the next screen composition. The Epoch Start DS is provided 
at a point in the AV Clip to which a skip operation may be performed, such as a start of a chapter in a movie. 
[0040] The Acquisition Point DS provides a display effect "display refresh", and relates to the preceding Epoch Start 
DS. There are two types of Acquisition Point DS: "Duplicate" and "Inherit". The Duplicate type DS is a completely 
identical DS to the preceding Epoch Start DS. The Inherit DS inherits the functional segments of the preceding Epoch 
25 Start DS but has different button commands. Although not being a start of an Epoch, the Acquisition Point DS contains 
all functional segments needed for the next screen composition. Thus, when the reproduction is started from an Ac- 
quisition Point DS, the graphics are reliably presented. That is to say, the Acquisition Point DS enables a screen 
composition from a midpoint in the Epoch. 

[0041] The Acquisition Point DS is provided in a point to which a skip operation may be made. Examples of such 

50 points include one that may be designated by a time search. The time search is an operation of locating a reproduction 
point corresponding to a time inputted by a user in minutes/seconds. Since the user input is made in a relatively large 
unit such as ten minutes and ten seconds, searchable reproduction points are located at intervals of 10 minutes or 10 
seconds. By providing the Acquisition Point DS In such a point searchable by a time search, the graphics stream can 
be smoothly presented when a time search is conducted. 

35 [0042] The Normal Case DS provides a display effect "display update", and contains only a difference from the 
previous screen composition. For example, if a button defined by a DS (v) has the same graphical representation as 
a button defined by an immediately preceding DS (u), but has a different state control. In this case, the DS (v) contains 
ICS or ICS and PDS only, and serves as a Normal Case DS. With this arrangement, the Normal Case DS is not required 
to contain overiapping ODSs, which leads to the reduction in the amount of data stored on the BD-ROM. Since the 

40 Normal Case DS only contains the difference, graphics cannot be displayed with the Normal Case DS alone. 

[0043] The interactive display defined by the above-described DSs presents GUI elements. The Interactivity of DSs 
refers to the capability of changing the state of each GUI element in accordance with a user operation. In this embod- 
iment, GUI elements Interacted by user operations are referred to as buttons. Each button has the normal state, the 
selected state, and the activated state. Each button state is rendered using a plurality of pieces of uncompressed 

45 graphics data, whbh is referred to as a "graphics objecr . One state of one button is associated with a plurality of 
graphics objects for presentation In animation. 

[0044] Now, a description ts given to Definition Segments (CDS: Object Definition Segment and PDS: Palette Defi- 
nition Segment), ii.;... 

[0045] The ODS is information defining a graphk^s object, which will be described later. Since AV Clips recorded on 
50 a BD-ROM feature its high image quality comparable to high-definition television, the graphics objects have a high 
resolution of 1920 x 1080 pixels. The color of each pixel is defined by 8-bit long index values showing a red color 
difference component (Cr value), a blue color difference component (Cb value), a luminance component (Y value), 
and a transparency (T value). This structure allows each pixel to be set to one of arbitrary 256 colors out of 1 6,77V,21 6 
colors. 

55 [0046] The ODS has a data structure shown in FIG. 1 0A. As shown in the figure, the ODS is composed of the following 
f iekis: "segment_type" showing that the type of this segment is an ODS; "segment Jength" showing the data length of 
the ODS; "objectjtf* uniquely identifying a graphics object within the EpOch, associated with the ODS; 
"object_version^number'' showing the version of the ODS within the Epoch; "last Jn_sequence_f lag"; and 
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"object__data_f ragment" carrying a consecutive sequence of bytes corresponding to part or ail of the graphics object. 
[0047] The objectjd field uniquely identrfies a graphics object within the Epoch, associated with the ODS. In the 
case where a plurality of graphics objects defined by a plurality of ODSs constitutes a sequence of animation, these 
ODSs are assigned serial objectjd values, 

5 [0048] Referring now to the Iastjn_sequence_flag field and the object_data__fragment field in more detail. Due to 
the constraint on a payload of PES packet, there may be a case where a piece of uncompressed graphics data con- 
stituting one button can not be carried by a single ODS. If this is the case, the graphics data is fragmented and each 
fragment is defined by an ODS in the object_data_fragment field. Here, every fragment except the last fragment is of 
the same size. That fe, the last fragment is less than or equal to the size of the preceding fragments. The ODSs carrying 

10 these fragments of the graphics object appear in the DS In sequence. The lastjn_sequence_flag field indicates an 
end of the graphics object. Although the above ODS data structure is based on a method of storing fragment in 
consecutive PES packets without a gap, the fragments may instead be stored in PES packets so as to leave some 
gaps between the PES packets. This concludes the description of ODS. 

[0049] Next, the PDS is explained. The PDS is Infonnation defining a palette for color conversion. FIG. 1 0B shows 
15 a data structure of the PDS. As shown in the figure, each PDS is composed of the following fields: " segment__type" 
showing, when set to the value "0x15", that the type of this segment is a PDS; "segmentjength" showing the data 
length of the PDS; "palletjd" uniquely identifying the pallet contained in the PDS; "palette.version _number" showing 
the version of the PDS within the Epoch; and "pallet_entry" showing the Color Difference Red {Cr_value), the Color 
Difference Blue (Cb_value), Luminance (Y^value), and Transparency (TLvalue). 
20 [0050] Next, an ICS Is explained. The ICS Is a functional segment defining the composition of an interactive display. 
The ICS has a data structure shown in FIG. 11. As shown in the figure, the ICS is composed of the following fields, 
"segment_type", "segment Jength", "composition_number", *'composition_state*', ''command_update_flag", 
**composition_time__out_pts", •'selection„time„out_pts", "UO_mask_table'', "animation_frame_rate_code'*, 
"defautt^selected^button^number"*, *'default_activated_button_number'*, and a group of "button info (1), (2), (3)". 
25 [0051 ] The composition_number field is set to a value out of 0-1 6 showing that the DS to which the ICS belongs is 
updated. 

[0052] The composition_state field shows that the DS beginning with the ICS is the Nomial case DS. the Acquisition 
Point DS, or the Epoch Start DS. 

[0053] The command_update_flag field indicates that the button commands in the ICS are changed from those 
30 defined in the previous ICS. For example, the DS to which an ICS belongs is an Acquisition Point DS, the ICS Is 
normally equal to those defined in the immediately previous ICS. Yet, by setting the command__update_flag to ON, the 
ICS can define different button commands from those defined in the immediately previous ICS. The 
command„update_fiag is set to ON when different button commands are associated with the same graphics objects. 
[0054] The composition_time„out_pts field describes the end time of the interactive display. At the end time, the 
55 interactive composition is no longer valid and thus no longer displayed. The composition_time_out_pts is preferably 
described with the frame accuracy of the video stream on the reproduction time axis. 

[0055] The selection_time_outj3ts field describes the termination time of the valid button selection period. At the 
time of the selection_time_out_pts, the button specified by the default_activated_button_n umber is activated. The 
value of the selection_t!me_out_pts is less than or equal to the value of the composition_time_out_pts. The 
40 selection_tlme_.out_pts Is described with the frame accuracy of the video stream on the reproduction time axis. 

[0056] The UO„mask_table determines permissions /prohibitions of User Operations for the duration of DS to which 
the ICS belongs. When this field is set to "Prohibited", a corresponding User Operation to the reproduction apparatus 
is invalid. 

[0057] The animation_frame„rate„code field specifies the frame rate to be applied to animated buttons. The anima- 
45 tion frame rate is given by the video frame rate divided by the value of the animation_frame_rate__code field. When 
this field is set to "00", only the graphics object designated by the start_objectJd_xxx is displayed for every button in 
non-animated state. The default_selected_button„number field indicates the button number that is selected as default 
when the presentation of the interactive display begins. When this field is set to the value "0", the button specified by 
the button number stored in a register of the reproduction apparatus is automatically activated. On the other hand, 
50 when this field is set to a value other than "0", the value indicates a valid button number. 

[0058] The default_activated_button_number field indicates the button Which is automatically activated when no 
button is activated by the user before the time defined by the seiection__time„out„pts field. When this field is set to 
"FP, the currently select button is activated at the time defined by the seiection_time_out _pts field. When this field is 
set to "00**, no button is automatically activated. If this field is set to a value other than "FF" and "OO", the value is 
55 interpreted as a valid button number. 

[0059] The button„info field provides Information defining buttons to be presented on the interactive display In the 
figure, the doted lines hp1 indicate that a data* structure of button Jnfo (1) is excerpted to be shown in detail. The 
buttonjnfo (i) carries information about the button (i) defined by the ICS. Hereinafter, a description is given to the 
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information items constituting the button Jnfo (I). 

[0060] The button_number field indfcates the value uniquely identifying the button (i) within the ICS. 
[0061] The numerically_selectablejlag field indicates whether the button (i) can be numerically selected. 
[0062] The auto_actionjlag field indicates whether to automatically activate the button {»). When the auto„action_flag 
is set to ON (bit value of "1"), the button (i) is transferred not to the selected state but directly to the activated state. 
On the other hand, when the auto_actionJlag is set to OFF (bit value of "0")^ the button (i) is transfen-ed not to the 
activated state but to the selected state when selected by a user. 

[0063] The button_hor!2ontal^osition field and button_vertical_position field respectively specify the horizontal and 
vertical positions of the top left pixel of the button (i) in the interactive display. 

[0064] The upper_button_number field specifies the button number of a button to receive the selected state at a 
push of the Move Up key while the button (i) is in the selected state. If this field is set to the same button number as 
that of the button (i), the user operation to the Move Up key is ignored. 

[0065] Similariy, the lower„button„n umber field, the left_button_number field, the right„button_number field specify 
the button number of a button to receive the selected state at a push of the Move Down key, Move Left key, and Move 
Right key, respectively, while the button (i) is in the selected state. If these fields are set to the same button number 
as that of the button (i), the user operations to the respective keys are ignored. 

[0066] The start_objectJd_normal specifies the first one of objectjds assigned serially to a group of ODSs consti- 
tuting the animation of button (I) In the normal state. 

[0067] The end_obJect Jd^nonmal field specifies the last one of the objectjds assigned serially to the group of ODSs 
constituting the animation of button (i) In the normal state. If the end_object_td_nonnal field specifies the same value 
as that of the start_object_ld_normal, the static image of the graphics object identified by the value is presented as the 
button (f). 

[0068] The repeat„normaLflag field specifies whether the animation of the button (i) in the nomnal state is to be 

continuously repeated. 

[0069] The start_objectJd_seiected field specifies the first one of objectjds assigned serially to a group of ODSs 
constituting the animation of button (i) in the selected state. If the end„objectJd_selected field specifies the same 
value as that of the start_objectJd_selected, the static image of the graphics object identified by the value is presented 
as the button (i). 

[0070] The end_objectJd„selected field specifies the last one of the objectjds assigned serially to the group of 
ODSs constituting the animation of button (i) in the selected state. 

[0071 1 The repeat„selected Jlag" field specifies whether the animation of the button (i) in the selected state is to be 
continuously repeated. 

[0072] The start„objectJd_activated field specifies the first one of objectjds assigned serially to a group of ODSs 
constituting the animation of button (i) in the activated state. 

[0073] The end__objectJd_activated field specifies the last one of the objectjds assigned serially to the group of 

ODSs constituting the animation of button (i) In the activated state. 

[0074] Next, a description is given to button commands. 

[0075] Each button„command is executed when the button (i) is activated. 

[0076] The button commands may include one instructing the reproduction apparatus to reproduce a PL or Playltem. 
Such a command Instructing ttie reproduction apparatus to reproduce a PL and Playltem Is referred to as a LinkPL 
command. When this command is executed, reproduction of the PlayList specified by the first argument is started from 
the point specified by the second argument. 

Format: LinkPL (first argument, second argument) 

[0077] The first argument is the PL number specifying the PL to be reproduced.. The second argument specifies, as 
a reproduction start point, a Playltem, a Chapter, or a Mark within the PL. 

[0078] The LinkPL function for specifying a Playltem as a reproduction start point is "LinkPL at Playltem {)", 
[0079] Tfie LinkPL function for specifying a chapter as a reproduction start point is "LinkPL at Chapter ()". 
[0080] The LinkPL function for specilying a mark as a reproduction start point is "LinkPL at Mark ()". 
[0081] The button commands may also include a command instructing the reproduction apparatus to obtain or set 
the state of the apparatus. The state of reproduction apparatus is shown by 64 player status registers (whose values 
are referredtoasPSRs) and 4, 096 general purpose registers (whose values are referred to as GPRs) . With the following 
commands (i) - (iv), specific values are set to the registers or the values ot the registers are obtained. 
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(i) Get vaiue of Player Status Register Command 
Format Get value of Player Status Register (argument) 

5 [0082] This function returns a value of the player status register specified with the argument. 

(ii) Set Value to Player Status Register Command 

Format: Set value of Player Status Register (first argument, second argument) 

10 

[0083] This function sets a value of the player status register specified with the f iiBt argument to the second argument, 
(lii) Get value of General Purpose Register Command 
15 Format: Get value of General Purpose Register (argument) 

[0084] This function retunes a value of the general purpose register specified with the argument, 
(iv) Set value to General Purpose Register Command 

20 

Format: Set value to General Purpose Register (argument) 

[0085] This function seta a value of the general purpose register specified with the first argument to the second 

argument. 

25 [0086] This concludes the description of the ICS stmcture. Next, a description is given to a specific exampie of 
interactive control defined by the ICS. The example relates to the ODSs and ICS shown in FIG. 12. FIG. 12 is a view 
showing the relation bietween the ODSs and ICS contained within a DS (n). The DS (n) includes the ODSs 11-19, 
21-29, 31-39, 41-49. The ODSs 11-19 are for rendering a button A in each of the three states. The ODSs 21-29 are 
for rendering a button B in each of the three states. The ODSs 31-39 are for rendering a button C in each of the three 

30 states. The ODSs 41 -49 are for rendering a button D in each of the three states (See the right brackets in the figure). 
The buttonjnfo (1), (2), (3), and (4) in the ICS provide the descriptions of state control of the buttons A-D (See arrows 
bh1 , bh2, bh3. and bh4 In the figure). 

[00871 Suppose that the control defined by the ICS is to be executed In synchronism with the display timing of a 
picture pti included in the video stream shown in FIG. 13. In this case, an interactive display tm1 composed of the 
35 buttons A-D is overiaid (gsl ) with the picture pti to produce a composite screen gs2. In this way, the interactive display 
composed of multiple buttons is presented in synchronism with a specific video image. Thus, the ICS makes it possible 
to present buttons In a way more real to the users. 

[0088] FIG. 14 shows an example of the description realizing the state transition of buttons A-D as illustrated in FIG. 
15. In FIG. 15, the arrows hhl and hh2 each visually represent the state transition defined by the neighborjnfo () 

40 within the button info (1). In the neighborjnfo (), the lower„button_n umber field specifies the button C. Consequently, 
a user operation to the Move Down key (up1 in the figure) while the button A Is in the selected state, the button C 
receives the selected state (sj1 In the figure). Similarly, the right_button_number field specifies the button B. Conse- 
quently, a user operation to the Move Right key (up2 in the figure) while the button A is in the selected state, the button 
B receives the selected state (sj2 in the figure). 

45 [0089] The arrow hh3 in FIG. 15 visually represent the state transition defined by the neighborjnfo Q within the 
button info (3). In the neighborjnfo 0» the upper„button„n umber field specifies the button A. Consequently, a user 
operation to the Move Up key (up3 in the figure) while the button C is in the selected state, the button A is put back 
into the selected state. . t r . 

[0090] Next, a description is given to graphical representations of the buttons A-D. The ODSs 11, 21. 31 , and 41 
50 each present images shown in FIG. 16. The ODSs 11-19 associated with the button A present images shown in FIG. 
1 7. The start_object Jd nomnal and end„objectJd nomnal fields specify the ODSs 11: and 1 3, respectively. Consequent- 
ly, the button A in the normal state is presented in animation by the sequence of the ODSs 11-13. In addition, in the 
selected__stateJnfo () of the buttonjnfo (1). the start_objectJd„selected and end_objectJd_seIected fields specify 
the ODSs 14 and 16, respectively. Consequently, the button A in the selected state is presented in animation by the 
55 sequence of ODSs 14-16. When a user puts the button A into the selected state, the images presented for the button 
A is changed from the ODS 11 -13 to the ODSs 14-16. By setting the repeat^nonrjaljlag and repeat_selectedjlag to 
the value "I", the animation of the ODSs 11-13 and of the ODSs 14-16 are continuously repeated, as represented in 
the figure with A", "A-^", B" and "B 
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£0091 ] As described above, each of the buttons A-B is associated with of a sequence of ODSs which can be presented 
in animation. Thus, with the ICS describing the control as above, such button state control is realized that a character 
image serving as a button changes its facial expression in response to user operations. 

[0092] Next, a description is given to application of the„numericaliy_.selectable_flag field. FIG. 18 is a view showing 
5 an example of the ICS and ODSs included in a DS. The ODSs 31-33 represents images of three baseball players 
along with their names and player numbers, as shown in the upper part of the figure. The ICS belonging to the DS 
includes three pieces of button infonnation, buttonjnfo (1), (2), and (3). The start_objectJd fields of buttonjnfo (1), 
(2), and (3) specify the ODS 31, OPS 32, and ODS 33, respectively. Similarly, the button _number fields of the 
buttonjnfo (1), (2), and (3) are set to the values of "99", "42*, and "94", respectively. Here, the 
numertcally_selectable_fiag fields of the buttonjnfo (1), (2), and (3) are all set to the value "I". With the above ar- 
rangement, each button defined by buttonjnfo (1), (2), and (3) is numerically selectable. When a user Inputs the value 
"99" on the remote controller 400, the button presented with the image of Mr. Beginner*s Luck receives the selected 
state. The value "99" may be inputted by pressing the "4" key and "9" key in a row or by pressing the "9" key once and 
"+1 0" key four time in a row. Alternatively, when a user inputs the value "42", the button of Mr. Careless Mistake receives 
15 the selected state. When a user inputs the value "94", the button of Mr. Dead Stock receives the selected state. 

[0093] Here, suppose the auto_actionJlag fields of the buttonjnfo (1), (2), and (3) are set to the value "1". In this 
case, the above three buttons receive, when selected, the active state instead of the selected state, and the button 
commands (LinkPL (PL#21), LinkPL (PL#22), LtnkPL (PL#23)) included in the buttonjnfo are executed. As a result. 
PL #21, #22, or #23 to which the executed button command is linked is reproduced. If those PLs define scenes of 
^ batting or pitching by the above players, each of these scenes is reproduced at an input of the numeric value corre- 
sponding to one of the player numbers. Since the buttons are sefectable with an input of the player numbers, whk^h 
are easy for the users to recognize, there is provided Improved user operabtlity. 

[0094] Next, a description Is given to the order of ODSs in the DS. As already described above, the ODSs belonging 
to one DS are associated by the ICS with each state of one button. The order of ODSs in the DS is determined in 

25 accordance with the button state with which each ODS is associated. 

[0095] In more detail, the ODSs belonging to one DS are grouped into (1) ODSs for presentation of a button in the 
normal state, (2) ODSs for presentation of the button in the selected state, and (3) ODSs for presentation of the button 
in the activated state. Each group for presentation of a respective state is referred to as a "button-state group". The 
button-state groups are arranged in the order, for example, of the normal state — > the selected state the activated 

30 state. As above, the order of ODSs within the DS is determined according to the button state with which the ODSs are 
associated. 

[0096] FIG. 1 9 Is a view showing the order of ODSs within a DS . In the figure, the following three button-state groups 
within the DS are shov/n on the second level: the button state group for presentation of the normal state (ODSs for 
Normal state) ; the button state group for presentation of the selected state (ODSs for Selected state) ; and the button 
35 state group for presentation of the activated state (ODSs for Activated state). In the figure, the button-state groups are 
arranged in the order of the normal state the selected state — > the activated state. This order Is determined so that 
the reproduction apparatus first reads the interactive composition constituting the first interactive display, and then 
reads the interactive composition to be presented only after update. 

[0097] FIG. 1 9 shows on the first level tJie graphics objects An, Bn, Cn, Dn, As, Bs, Cs, Ds, Aa, Ba, Ca, and Da that 
-#0 are referenced by the button-state groups. The numerical subscript /? as in An, Bn, Cn, and Dn Indicates a respective 
button in the nomiai state. Similariy, the numerfcal subscript s as in As, Bs, Cs, Ds Indicates a respective button in the 
selected state, and the numerical subscript a as in Aa, Ba, Ca, and Da Indicates a respective button In the activated 
state. The figure shows on the second level the button-state groups to whk^h the graphk^s objects shown on the first 
level belong. In the figure, there are more than one ODS appended with the numerical subscripts, such as "I" and" n\ 
45 to read "ODS 1" and "ODS rf. Yet, it should be noted that each ODS 1 included in N-ODS, S-ODS andA-ODS is 
different. The same holds to the drawings with similar reference numerals. 

[0098] FIG. 20 is a view illustrating the button state transition on the interactive display defined by the button-state 
groups shown in FIG. 19. 

[0099] As shown in the figure, the interactive display has a plurality of states including "initial state**, "display update 
so by 1 st user action", and "display update by 2nd user action**. The arrows In the figure represent user actions triggering 
the state transition. The four buttons A, B, C, and D each have the normal =state, the selected state, and the activated 
state. In order to present the initial interactive display, the graphics objects for the three buttons in the normai state 
and one button in the selected state need to be ready for presentation. - ■ - . 

[0100] When a default selected button is not specified, it is not dete mn in ed which of the buttons A-D will be selected 
55 first. Even so, the initial interactive display can be made upon completion of decoding the graphic objects for the nonnal 
and selected states of each button. In view of this observation, the button -state groups in this embodiment are arranged 
in the order of the normal state —> the selected state ^ the activated state, as shown in FIG. 1 9. With this arrangement, 
the initial interactive display can be presented even if the ODSs for the activated state are not yet read and decoded. 
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As a result, the duration is shortened between the reading of DS is comnrtenced and the presentation of initial interactive 
display is connpleted. 

[0101 1 With reference to FfG. 21 , a description is given to the order of ODSs that are shown in FIGs. 1 6 and 1 7. FIG. 
21 is a view showing the order of ODSs within a DS. In the figure, the ODSs for norma! state include ODSs 11-13, 

5 21-23, 31-33, and 41-43. The ODSs for selected state include ODSs 14-16, 24-26, 34-36, and 44-46. The ODSs for 
activated state includes ODSs 17-19, 27-29, 37-39, and 47-49. The ODSs 11-13 are for animated presentation of the 
character image shown in FIG. 17. Similarly, the ODSs 21 -23, 31 -33, and 41 -43 are for animated presentation of other 
character images. Thus, by placing the button-state group composed of the above ODSs (ODS 11-13, 21 -23, 31-33, 
41-43} as the first button state group In the DS, the initial interactive display Is ready for presentation even before the 

to reading of the DS is completed. This ensures that the interactive display composed of animated buttons is displayed 
without delay. 

[0102} Next, a description is given to the order of ODSs when there are multiple references thereto from a plurality 
of button-state groups. The term multiple references means that the same objectjd is referenced by two or more 
pieces of normaLstate Jnfo, selected_state_info, and act}vated_stateJnfo within the ICS. With the multiple references, 

15 for example, a specific graphics object used to render a button in the nonnal state is commonly used to render another 
button in the selected state. In other words, the graphics object is shared, so that the number of ODSs can be reduced. 
Here, a problem arises as to which button-state group an ODS with multiple references belongs. Specifically, when 
one ODS is associated with one button in the nomial state as well as another button in the selected state, it needs to 
be detennined whether the ODS belongs to the button-state group for the nonnal state or the button-state group for 

^0 the selected state. In such a case, the ODS is placed in the button-state group that first appears. For example, if the 
button-state groups appear in the order of the normal state the selected state -> the activated state, an ODS refer- 
enced by both the nonnal-state and selected-state groups is placed in the normal-state group. Also, an ODS referenced 
by both the selected-state and activated-state groups is placed in the selected-state group. This concludes the de- 
scription of the order of multiple referenced ODSs. 

25 [0103] Now, a description is given to the order of ODSs within the button-state group for the selected state. In the 
button-state group for the selected state, which of the ODSs should be placed at the beginning depends on whether 
a default selected button is determined. The default selected button is specified when the 
default_selected_button_number field is set to a valid value other than "00*. In this case, the ODS associated with the 
default selected button is placed at the beginning of the button-state group. 

30 [01 04] No default selected button is specified when the default selected button„number field In ICS is set to the value 
"00". The default_selected„button_number field is set to the value "00" in the case, for example, where an AV Clip into 
which the DS is multiplexed serves as a merge point of a plurality of reproduction paths. Suppose that the preceding 
reproduction paths correspond to Chapters 1. 2, and 3, a DS serving as a merge point is for presentation of buttons 
associated with Chapters 1, 2, and 3, In such a case, no specific button can be specified in the 

35 default_selected_button_number field. 

[0105] Ideally, when the interactive display is presented after reproduction of Chapter 1 , the button associated with 
Chapter 2 should be selected as default. Similarly, after reproduction of Chapter 2, the button associated with Chapter 
3 should be selected as default, and the button associated with Chapter 4 after reproduction of Chapter 3. In other 
words, the default_selected„button_number field is set to the value "0" to be invalidated when a different button needs 

40 to be selected as default depending on which reproduction path has been taken. In this case. It is not necessary to 
place a specific button-state group ODSs at the beginning because no specific button is selected as default. 
[0106] This concludes the description of the order of ODSs. Next, a description is given to how DSs having above- 
described ICSs and ODSs are allocated on the reproduction time axis of AV Clip. An Epoch is a time period on the 
reproduction time axis during which memory management is continuous. Since each Epoch is composed of one or 

45 more DSs, it is important to how to allocate those DSs on the reproduction time axis of AV Clip. The reproduction time 
axis of AV Clip mentioned here is a time axis for defining decoding timing and presentation timing of individual pictures 
which constitute the video stream multiplexed in the AV Clip. The decoding timing and presentation timing on the 
reproduction time axis are expressed with a time accuracy of 90 KHz, This time accuracy of 90 KHz corresponds to a 
common multiple of NTSC signal, PAL signal, Dolby AC-3, and the frame frequeri<^ of MPEG audio. A DTS and a PTS 

50 attached to ICS and ODS within a DS specify timings on this reproduction time axis for achieving synchronous control, 
in other words, the DSs are allocated on the reproduction time axis by exercising synchronous control using DTS and 
PTS attached to ICS and ODS. • ' 

[0107] First, a description is given to the synchronous control exercised using DTS and PTS of an ODS. 
[01 08] The DTS shows a time, with the time accuracy of 90 KHz, at which Itie decoding of ODS needs to be started. 
55 The PTS shows a deadline for completing the decoding. 

[0109] The decoding of ODS can not be completed instantly, and takes a certain duration. In order to explicitly show 
the start and end times of decoding the ODS, the DTS and PTS of the ODS show the decoding start time and decoding 
deadline. ^- 
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[0110] The value of PTS shows the deadline by which the decoding of ODS (j) needs to be completed and the 
resulting unconnpressed graphics object needs to be available in the object buffer of the reproduction apparatus. 
[0111] The decoding start time of an arbitraryODS (J) belonging to a DS (n) is shown by DTB (DSn [ODS}) with the 
time accuracy of 90 KHz. Thus, the decoding deadline of the ODS (j) is determined by adding to the DTS (DSn [CDS]) 

.5 value, a maximum time which may be taken for the decoding. 

[0112] Let SIZE (DSn[ODSyi) denote the size of ODS G), and Rd denote the ODS decoding rate. Then the maximum 
. time required for the decoding (in seconds) is SiZE(DSn[ODS/])//Rd, 
[01 13] By converting this maximum time to the accuracy of 90 KHz and adding the result to the DTS of the CDS (j), 
the decoding deadline to be shown by the PTS is calculated with the accuracy of 90 KHz. 

10 [0114] The PTS of the ODS Q) belonging to the DS (n) can be expressed by the following fonnula: 

PTS (DS[ODS/I) - DTS(DSn [ODS/J) + 
^5 90.000 X (SIZE(DSn[ODS/I) // Rd) 

[01 15] Next a description is given to the value of PTS within an ICS. 

[01 16] The PTS within the ICS Is set to the value obtained by adding (1 ) the PTS value of the ODS of which decoding 
time is latest among all the ODSs needed for the Initial display of the DS (n), (2) the time taken to clear the graphics 

20 plane, and (3) the time taken to transfer the decoded graphics object to the graphk^s plane. 

[0117] When the default_seiected_button_number is specified in the ICS, the initial interactive displayed can be 
presented upon completion of the decoding of ODSs associated with each button in the normal state and the specified 
default selected button in the selected state. The ODSs for rendering each button in the selected state are referred to 
as S-ODSs, and the S-ODS of which decoding start time is earliest (i.e. the ODS for rendering the default selected 

25 button, In this case) is referred to as S-ODSsfirst. The PTS value of this S-ODSsfirst is designated as the PTS value 
of the ODS of which decoding start time is latest, and used as a reference value for the PTS within the ICS. 
[0118] When the default_selected„button_number is not specified in the ICS, It is not known which of the buttons 
willbefirstselected. Thus.theinitialinteractive display is not available for presentation until it is ready to render every 
button in the normal state as well as the selected state. Among the S-ODSs for rendering the selected state of each 

30 button included in the initial interactive display, the ODS of which decoding start time Is latest is referred to as the 
S-ODSslast The PTS value of this S-ODSslast is designated as the PTS value of the ODS of whfch decode start time 
is latest, and used as a reference value for the PTS within the ICS. 

[0119] Let PTS (DS/i[S-ODSsf'trst)] denote the decoding deadline of S-ODSsfirst, then PTS (DSn[lCS]) is set to the 
value obtained by (2) the time taken to clear the graphic plane and (3) the time taken for rendering the decoded graphics 

35 object to the graphics plane. 

[0120] Let video_width and video_height denote the width and height of a rectangular area of graphics plane within 
which graphics objects are rendered. When the rendering rate to the graphics plane is 128 Mbps, the lime taken to 
clear the graphics plane is obtained by 8 x video_width X video_.height // 128,000,000. With the time accuracy of 90 
KHz, the above-mentioned time (2) taken to clear the graphics plane is expressed by 90,000 X (8 x video„width x 

40 video_height // 1 28, 000, 000) . 

[0121] Let ISIZE(DSn[ICS.BUTTON[/l] denote the total size of graphics objects specified by each piece of button 
information contained in the ICS. When the rendering rate to the graphics plane is 128 Mbps, the time taken to render 
the graphics plane is obtained by ZSIZE(DSn[ICS. BUTTON [/]]) // 128.000,000. With the time accuracy of 90 KHz. the 
above-mentioned time (3) taken to render the graphics plane is expressed by 90, 000 x (i:siZE(DSn[ICS.BUTTON 

45 [/]])// 128,000,000). 

[0122] With the above fomnulas, PTS (DSn [ICS]) is expressed by the following fomnula: 

PTS(DSn[ICSj) ^ PTS {DSn [S-ODSsfifst]) + ' 

50 . 

90,000 X (8 x video_width x video_h eight // 128, 000, 000) 
+ 90,000 X (XSIZE(DSn[tCS.BUTTON[/|]) // 128,000,000) 

55 [0123] Note that the above formula is applicable when the default_selected_button_number in the ICS is valid. If 
not, the following fonnula needs to be satisfied. 
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PTS(DSn{ICS]) > PTS(DSn[S-ODSslast]) -f 
90,000 X (8 X video„width x video_height//1 28,000,000) + 
^ 90,000 X (SSIZE(DSn[ICS.BUTTON[/11)//1 28,000,000) 

[0124] By setting the PTS and DTS in the above manner, synchronous display is realized. FIG. 22 Illustrates an 
example of such control. In this example, buttons are to be displayed tn synchronization with the display timing of a 
10 picture py1 in the video stream. To this end, the PTS within the ICS needs to be set to the value corresponding to the 
display point of picture data pyl . Note that the DTS within the ICS Is set to show the time prior to the time shown by 
the PTS. 

[0125] The decoding of ODSs constituting the initial interactive display of DS (n) needs to be completed by the time 
calculated by subtracting from the time shown by the PTS in the ICS, the plane clear duration cd1 and the object 

1^ transfer duration tdl. Thus, the PTS of the ODS of which decoding time Is the latest needs to be set to the value 
corresponding to a point of time marked in the figure with a black star. Further, a duration dd1 is taken for decoding 
the ODS, so that the DTS of the ODS needs to be set to a time earlier than the PTS by the duration dd1 . 
[0126] The example illustrated in FIG. 22 is a simplified case in that only one ODS is used for overlay with the video 
data. In the case where a plurality of ODSs are used for presentation of the initial interactive display, the PTS and DTS 

20 of the ICS and of ODS need to be set as illustrated in FIG. 23. 

[0127] FIG. 23 is a view illustrating the DTS and PTS setting in the case where the initial interactive display Is con- 
structed by a plurality of ODSs and a default selected button is specified. Suppose that the duration ddi is taken for 
decoding S-ODSsfirst, which is the ODS of which decoding time is latest among all ODSs needed for presentation of 
the initial interactive display. The PTS (DSrj[S-ODSsfirst]) of this S-ODSsfirst is set to the value corresponding to the 

25 end of the duration dd1 . 

[0128] Further, for presentation of the initial interactive display, the graphics plane must be cleared and the decoded 
graphics objects must be transferred. Thus, the PTS (DS/7[ICS]) of the ICS must be set to the value corresponding to 
the time, at the earliest, calculated by adding to the value of PTS(DS/^S-ODSsfirst]), the plane clear duration (90,000 
X (8 X video_width x video__height//1 28,000,000) and the decoded object transfer duration (90,000 X CESIZE(DS/? 

30 [ICS.BUTTONMJ) // 1 28,000,000)). 

[0129] FIG. 24 is a view illustrating the DTS and PTS setting in the case where the initial interactive display Is con- 
structed by a plurality of ODSs and no default selected button is specified. Here, the PTS (DSn [S-ODSstast]) is set 
to the value corresponding to an end of a duration dd2 taken for decoding the S-ODSslast, which is the ODS of which 
decoding time is latest among all S-ODSs needed for presentation of the initial interactive display. 

35 [0130] Further, for presentation of the initial interactive display, the screen must be cleared and decoded graphics 
objects must be transferred. Tnus, the PTS (DSn[ICS]) of the ICS needs to be set to the value corresponding to the 
time, at the earliest, calculated by adding to the value of PTS(DSr^S-ODSs)ast3), the screen clear duration (90,000 x 
(B x video_width x video_height // 128,000,000) and the decoded object transfer duration (90,000 x (ZSIZE (DSn 
[lCS.BUTTON[/]]) // 128,000,000)). This concludes the description regarding the synchronization control defined by 

40 the ICS. 

[0131] In the case of DVDs, the interactive control is valid during the time of VOBU, which corresponds to GOP of 
the video stream. In the case of BD-ROMs, however, the valid period of interactive control can be arbitrarily set with 
the use of the PTS and DTS of the ICS included in an Epoch. That is to say, the interactive control of BD-ROM is 
independent of GOP 

45 [0132] Note that synchronous control by the PTS of the ICS includes not only displaying a button at a specific point 
on the reproduction time axis, but also making a Popup Menu available for presentation during a specific time period 
on the reproduction time axis. The Popup Menu is a menu displayed at a push of a menu key of the remote controller 
400. The Popup menu becomes available for presentation at the display timing of a specific picture within the AV Clip. 
Such control is also included in the synchronous control defined by the PTS of the ICS. Similarly to ODSs used for 

50 presentation of buttons, ODDS for presentation of the Popup menu are decoded and the decoded objects are rendered 
on the graphics plane. Unless the rendering to the graphfes plane is completed, no response can be made to a menu 
cail from users. In order to allow the synchronous display of the Popup menu, the PTS of the ICS shows the time upon 
which the Popup display becomes available. 

[0133] This concludes the description of the recording medium according to the first embodiment of the present 
55 invention. The following describes a reproduction apparatus according to the finst embodiment of the present invention. 
FIG. 25 shows an internal structure of a reproduction apparatus. The reproduction apparatus according to the present 
invention is industrially manufactured based* on this internal structure. The reproduction apparatus is roughly made up 
of three parts that are a system LSI, a drive device, and a microcomputer system. The reproduction apparatus can be 
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manufactured by mounting these parts on a cabinet and substrate of the apparatus. The system LSI is an integrated 
cifcuit Including various processing units for achieving the functions of the reproduction apparatus. The reproduction 
apparatus manufactured In the al:>ove manner is composed of a BD drive 1 , a track buffer 2, a PID fitter 3, transport 
buffers 4a, 4b. and 4c, a peripheral circuit 4d. a video decoder 5, a video plane 6, an audio decoder 7, a graphics plane 
s 8, a GLUT unit 9, an adder 10, and a graphics decoder 12, a coded data buffer 13, a peripheral circuit 13a, a Stream 
Graphics Processor 14, an object buffer 15, a composition buffer 16, a graphics controller 17, a UO controller 18, a 
player register group 1 9, and a controller 20. 

[0134] The BD drive 1 performs loading, reading, and ejecting of the BD-ROM so as to access the BD-ROM, 
[0135] The track buffer 2 is a FIFO memory. Accordingly, TS packets read from the BD-ROM are removed from the 
track buffer 2 In the same order as they arrive. 

[01 36J The PID filter 3 performs filtering on TS packets output from the track buffer 2. In more detail, the PID filter 3 
passes only TS packets having predetennlned PfDs to the transport buffers 4a, 4b» and 4c. There is no buffering 
required inside the PID filter 3. Accordingly, TS packets entering the PID filter 3 are written to the transport buffers 4a, 
4b, and 4c without delay. 

IS £0137] The transport buffers 4a, 4b, and 4c are FIFO memories for storing TS packets output from the PID filter 3. 

[0138] The peripheral circuit 4d has a wired logic for converting TS packets read from the transport buffers 4a, 4b, 

and 4c to functional segments. The functional segments are then stored to the coded data buffer 13. 

[0139] The video decoder 5 decodes TS packets output from the PID filter 3 to obtain uncompressed pictures, and 

writes the obtained pk^tures onto the video plane 6. 
20 [0140] The video plane 6 is a plane memory for video data. 

[01 41 ] The audio decoder 7 decodes TS packets output from the PID filter 3, and outputs uncompressed audio data. 

[0142] The graphics plane 8 Is a plane memory having a memory area of one screen, and is capable of storing 

uncompressed graphics of one screen. 

[0143] The GLUT unit 9 converts index colors of the uncompressed graphics on the graphics plane 8, based on Y, 
25 Cr, and Cb values defined in a PDS. 

[01 44] The adder 1 0 multiplies the uncompressed graphics converted by the GLUT unit 9, by a T value (transparency) 
defined in the PDS. The adder 1 0 then performs addition for corresponding pixels in the resulting uncompressed graph- 
ics and the uncompressed picture data on the video plane 6 to output a composite image. 

[0145] The graphics decoder 12 decodes a graphk^ stream to obtain uncompressed graphics, and renders the 
30 uncompressed graphics to the graphk:s plane 8 as graphics objects. As a result of decoding the graphk^ stream, 
subtitles and menus appear on the screen. This graphk^ decoder 12 is composed of the coded data buffer 13, the 
peripheral circuit 13a, the stream graphics processor 14, the object buffer 15, ttie composition buffer 16, and the graph- 
ical controller 1 7, 

[0146] The coded data buffer 13 is used for storing functional segments together with their DTSs and PTSs. The 

35 functional segments are obtained by removing a TS packet header and a PES packet header from each TS packet 
stored in the transport buffers 4a, 4b, and 4c, and arranging the payloads in sequence. The DTSs and PTSs contained 
in the removed TS packet headers and PES packet headers are stored in correspondence with the PES packets. 
[0147] The peripheral circuit 13a has a wired logic for transferring data from the coded data buffer 1 3 to the stream 
graphics processor 14, as well as from the coded data buffer 13 to the composition buffer 16. In more detail, when the 

40 current time reaches the DTS of an ODS, the peripheral circuit 13a transfers the ODS from the coded data buffer 1 3 
to the stream graphics processor 14. Also, when the current time reaches the time shown by the DTS of an ICS or of 
a PDS, the peripheral circuit 13a transfers the ICS or PDS from the coded data buffer 13 to the composition buffer 1 6. 
[0148] The stream graphics processor 14 decodes an ODS to obtain uncompressed graphics having index colors, 
and transfers the uncompressed graphics to the object buffer 15 as a graphics object The decoding by the stream 

45 graphics processor 14 begins at the time shown by the DTS associated with the ODS and ends by the decoding 
deadline shown by the PTS also associated with the ODS. The decode rate Rd of the graphics object decoding men- 
tioned above is equal to the output rate of the stream graphics processor 14: ; : . 
[0149] The object buffer 15 stores graphics objects decoded by the stream graphics processor 14. 
[0150] The composition buffer 1 6 is used for storing an ICS and a PDS. - : ^ 

so [01 51 ] The graphics controller 1 7 decodes an ICS stored in the composftion. buffer 16, and carries out control based 
on a decoding result at the timing specified by the PTS attached to the ICS; - ; 

[01 52] The UO controller 1 8 detects a user operation made on the remote controller and the front panel of reproduc- 
tion apparatus, and outputs information showing the detected user operation (hereinafter referred to as a UO (User 
Operations)) to the controller 20. - , . : . . 

55 [0153] The player register group 1 9 is a group of registers provided within the controller 20, and includes 32 player 
status registers and 32 general purpose registers. The meanings of player status register values (PRS) are shown 
below. Tfcre expression "PSR (x) " represents the value of the xth player status register. 
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PSR(O) : Reserved 

PSR(1): Audio Stream Number to be decoded 

PSR (2) : Subtitle Graphics Stream Number to be decoded 

PSR{3): Angle Number set by User 
5 PSR(4) : Title Number being Current Reproduction Target 

PSR(5): Chapter Number being Current Reproduction Target 

PSR(6): PL Number being Current Reproduction Target 

PSR(7): Piayltem Number being Current Reproduction Target 

PSR(8): Time Information showing Current Reproduction Point 
io PSR(9): Count of Navigation Timer 

PSR{10): Button Number Currently In Selected State 

PSR(11)-{12): Reserved 

PSR(1 3): Parental Level set by User 

PSR(14): Reproduction Apparatus Setting for Video 
15 PSR(15): Reproduction Apparatus Setting for Audio 

PSR(16): Language Code for Audio 

PSR{17): Language Code for Subtitles 

PSR(18): Language Code for Menu Description 

PSR (19) - (63) : Reserved 

20 

[01 54] The PSR (8) rs updated each time a picture belonging to the AV Clip is displayed. That is, when the reproduction 
apparatus displays a new picture, the PSR{8) is updated so as to hold the value corresponding to the display start time 
(i.e. Presentation Trme) of the newly displayed picture. With reference to PRS (8), the current reproduction point is 
known. 

25 [01 55] The controller 20 perfonrjs integral control through the bi-directional data exchanges with the graphics decoder 
12. The data outputted from the controller 20 to the graphics decoder 12 is UOs received by the UO controller 18. The 
data outputted from the graphics decoder 12 to the controller 20 is button commands included in an ICS. 
[0156] The components of reproduction apparatus having the above structure perform processing in a pipeline. 
[01 57] FIG. 26 is a timing chart showing decoding of ODSs. in the figure, the forth level shows DSs recorded on the 

30 BD-R01V1. The third level shows the read durations of ICSs, PDSs, and ODSs by the coded data buffer 13. The second 
level shows the decode durations of the ODSs by the stream graphics processor 14. The first level shows durations 
of processing by the graphics controller 17. The decoding start time of each CDS is shown in the figure as DTS11 , 
DTS12, and DTS13. Each CDS needs to be read to the coded data buffer 13 by the decoding start time specified by 
a respective DTS. Therefore, the reading of CDS 1 is completed, at the latest, immediately before the decode duration 

ss dpi of the CD SI to the coded data buffer 1 3. Similarty, the reading of ODS (n) is completed, at the latest, immediately 
before the decode duration dp2 of the ODS2. 

[0158] The decoding deadline of each ODS is shown in the figure as PTS11 , PTS12, and PTS13. The decoding of 
ODS1 by the stream graphics processor 14 is completed by the time shown by the PTS1 1 , and the decoding of ODS 
(n) is completed by the time shown by the PTS12. As above, each ODS is read to the coded data buffer 13 by the time 
40 . shown by a respective DTS of the ODS, and the read ODS is decoded and loaded to the object buffer 15 by the time 
shown by a respective PTS of the ODS. The above processing is performed in pipeline by the single stream graphics 
processor 1 4. 

[01 59] In the case where a default selectedbutton Is specified^ all of the graphics objects necessary for presentation 
of the initial interactive display become available on the object buffer 16 upon completion of the decoding of all of the 

4S ODSs for the normal state and the first ODS for the selected state. In the figure, at the time shown by the PTS13, all 
of the graphics objects necessary for presentation of the initial interactive display become available. 
[01 60] On the first level, the duration cdl is needed for the graphics controller 1 7 to clear the graphics plane 8, and 
the duration tdl is needed for rendering the graphics on the object buffer 1 5 to the graphics plane 8. The position within 
the graphics plane 8 for rendering is specified by the button_horizontal_position and button_vertical_position fields in 

50 the ICS. That is to say, uncompressed graphics constituting an interactive :display is obtained on the graphics plane 8 
at the end of time calculated by adding to the PTS13 value of the ODS, the plane clear duration cd1 and the rendering 
duration tdl of the decoded graphics objects. The CLUT unit 9 perfomns color conversion on the uncompressed graph- 
ics, and the adder 10 overtays the graphics on an uncompressed picture stored with the video plane 6. As a result, a 
composite image is produced. 

55 [0161] With the above arrangement, the initial interactive display is presented without waiting for completion of the 
decoding of the ODSs belonging to the button-state groups for the selected state and the activated state. Thus, in 
comparison with the case where the initial interactive display is presented only after ail the ODSs within the OS are 
decoded, the presentation is earlier by the duration hyl shown in the figure. 



16 



EP 1 608 165 A1 



[0162] Note in the figure, there are more than one ODS appended with the numerical subscripts, such as "I** and 
"n**, to read "ODS 1" and "ODS n". Yet. each ODS 1 included in N-ODSs, S-ODSs and A-ODSs is different. The sanne 
holds to the drawings with similar reference numerals. 

[01 63] In the graphics decoder 1 2, the stream graphics processor 1 4 continues decoding while the graphics controller 
5 17 is clearing or rendering the graphics plane 8 (the decode durations of ODS (n). ODS 1, ODS (n) shown on the 
second level) . Since the decoding of the remaining ODSs are continued even during the graphics controller 1 7 is 
clearing or rendering the graphics plane 8, the decoding of the remaining ODSs are completed eariier than would 
otherwise be the case. Thus, updates of the interactive display using those remaining ODSs can be carried out imme- 
diately upon a user operation. As a result of such pipeline processing, presentation of the initial interactive display and 
10 updates to the display can be carried out speedily. 

[0164] The example in FIG. 26 relates to the case where a default selected button is specified. FIG. 27 is a timing 
chart showing the pipeline processing perfomned by the reproduction apparatus in the case where no default selected 
button is specified. In such a case, all of the graphics objects needed for presentation the initial display become available 
upon completion of decoding all the ODSs for the normal state and for the selected state, and loading the resulting 
15 objects to the graphics plane 8. With this arrangement, the initial interactive display is presented without waiting for 
completion of the decoding of the ODSs for the activated state. Thus, in comparison with the case where interactive 
display Is presented only after all of the ODSs within the DS are decoded, the presentation is eariier by the duration 
hy2 shown in the figure. 

[01 65] This concludes the disruption of the internal structure of reproduction apparatus. Next, a description is given 
^0 to implementation of the controller 20 and the graphics decoder 1 2. The controller 20 can be implemented by a general- 
purpose CPU executing a program of the steps shown by FIGs. 28 and 29. Hereinafter, a description is given to op- 
erations performed by the controller 20 with reference to FIGS. 28 and 29. 

[01 66] FIG. 28 is a flowchart of the operations perfomned by the controller 20 for executing a LinkPL function. Upon 
decoding a command including a LinkPL function, the controller 20 operates as shown in FIG. 20. 

25 [0167] In this flowchart, the Playltem and Access Unit to be processed are denoted as PI (y) and Access Unit (v), 
respectively. As shown in the flowchart, the reproduction apparatus reads the PL (.mpls) specified by the argument of 
the LinkPL (step SI), and designates the first Pt in the current PL as a PI (y) (step S2), The reproduction apparatus 
then reads Clip infonnation specified by the Clip Jnformation_file_name field In the PI (y) (step S3). 
[0168] Upon reading the Clip infonnation, the reproduction apparatus converts the value of IN_time field within the 

30 PI (y) to an address using the EP_map Included in the Clip information (step S4). The Access Unit specified by the 
resulting address is designated as an Access Unit (v) (step S5). The reproduction apparatus also converts the value 
of Out_time field to the address using the EP„map included in the Clip infonnation (step S6). The Access Unit specified 
by the resulting address is designated as an Access Unit (w) (step S7). 

[0169] Once the Access Units (v) and (w) are designated, the reproduction apparatus instructs the BD drive to read 
35 the Access Units from (v) through (w) (step S8), and also instructs the video decoder 5, the audio decoder 7. the 
graphics decoder 1 2 to decode the data residing at the addresses corresponding to the ln_time through Out_time fields 
of the PI (y) (step S9) . 

[0170] In a step S1 1 , an end judgment of ttiis flowchart is made to see if the PI (y) is equal to PI (z) . If the step S11 
results in "YES", the processing of this flowchart is tenninated. If not, the next Playltem is designated as a new PI (y) 
40 (step S1 2), and the processing goes back to the step S3. The steps S1 -S1 0 are repeated until the step S1 1 results in 

[0171] In a step SI 0, a function segment is loaded to the coded data buffer 13 simultaneously with the reading of 

Access Units. 

[0172] FIG. 29 is a flowchart of the operations prefonned for loading functional segments. In the flowchart, a Segment 
45 (K) is a variable indicating a segment (ICS, ODS, or PDS) read with an Access Unit, and an ignore flag Indicates 
whether the Segment (K) is to be ignored or loaded. In this flowchart, after the ignore flag is initialized to 0 (step S20), 
a loop of steps S21 to S24 and S27 to S35 is perfomied for the Segment (K) (S25 and S26). 

[0173] In the step S21 , the reproduction apparatus judges whether the Segment (K) is an ICS. If the Segment (K) is 
an ICS, the processing goes to the judgments in the steps S27 and S28.-- 
50 [0174] In the step S27, a judgment is made as to whether the segment_type field within the ICS is set to a value 
indicating an Acquisition Point DS. If the Segment (K) belongs to an Acqufsition Point DS, the processing goes to the 
step S28. On the other hand, if the Segment (K) belongs to either an Epoch Start or Nornial Case DS, the processing 
goes to the step S33. ' *■ ' 

[0175] In the step 828, a judgment is made as to whether the immediately preceding DS Is stored in the coded data 
55 buffer 13. The step S28 is perfomied if the step S27 results in "YES". The immediately preceding DS is not stored in 
the coded data buffer 13 if a skip operation Is perfomed. In this case, the presentation needs to be started from an 
Acquisition Point DS, so that the processing goes to the step S30 (S28: NO). 

[0176] On the other hand, in the case where the immediately preceding DS is stored in the coded data buffer 13 
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(step S28: YES), the ignore flag is set to "I** (step S29), and the processing goes to the step S31. 
[0177] In the step S31 , a Judgment is made as to whether the command_update_flag field is set to "1 If set to "1 " 
(step S31: YES), the button command of the button information is loaded to the coded data buffer 13 while the other 
data is ignored (step S32) . If the command_update_flag field is set to "O". the processing goes to the step S22- Con- 
5 sequentiy, the SCS indicating an Acquisition Point is ignored (step 824) . 

[0178] With the ignore flag set to "1 the step 822 results in "NO", so that all the functional segments belonging to 
an Acquisition Point DS is ignored. 

[0179] in a step S33, a judgment is made as to whether the segment_type field within the ICS indicates a Normal 
Case DS. if the Segment (K) belongs to an Epoch Start DS» the ignore flag is set to "0" in the step S30. 

10 [0180] lftheignorefiagissetto"0"(StepS22: YES),theSegment(K)isloadedtothecodeddatabuffer13 (stepS23) , 
[0181] On the other hand. If the Segment (K) belongs to a Nonrial Case DS. the processing goes to the step S34, 
Slmiiarly to the step S28, In the step S34, a judgment is made as to whether the immediately preceding DS is stored 
in the coded data buffer 13. If the immediately preceding DS is stored, the ignore flag is set to "0" (step S30). If not, 
the ignore flag is set to T (step S35). With the ignore flag set as above, the functional segments belonging to a Nonnal 

^5 Case DS are ignored if the immediately preceding DS is not stored in the coded data buffer 13 of the reproduction 
apparatus. 

[0182] in the case where the auto_action_flag field in each piece of button information (1), (2), and (3) is set to "I", 
the three buttons go into the activated stated Instead of the selected state, and the button commands {UnlcPL(PL#21 ), 
LinkPL(PL#22). and LtnkPL(PL#23)) within the pieces of button infomiation are executed. Suppose, for example. 
PL#21 , PL#22 , and PL#23 to which the button commands are linked are butting and pitching scenes of baseball players, 
those scenes are reproduced upon a numeric input of a corresponding player number. Since the selection is made 
with an input of a player number, which is easy to recognize for users, the user operabHity is further enhanced. 
[0183] Now, a description is given to how DSs are read in the example shown in FIG. 30. In this example, three DSs 
(DS1 , DS10, and DS20) are multiplexed with video data, in the DS1 , which is the first of the three DSs, thesegment„type 
25 field shows Epoch Start, the command„update_flag field is set to "0", and the button command UnkPL(PL#5) is in- 
cluded. 

[0184] The DS 10 is a duplicate of the DS1 . In the DS10, the segment_type field is set to a value indicating an 
Acquisition Point DS, the command_update_flag is set to "0**, and the button command LinkPL(PL#5) is included. 
[0185] The DS20 is inherited from the DS1 and Its segment_type field value indicates an Acquisition Point DS. The 
50 DS20 differs from the DS1 in the button command (LlnkPL(PL#1 0)), so that the connmand_update_f lag is set to "1 " to 
signal that the button command is different. 

[01 86] Suppose a skip operation is made to a picture pti 0 in the AV Clip in which the above three DSs are multiplexed 
v/ith video data, in such a case, the DS10 v/hich is closest to the skip destination point is subjected to the processing 
shown in FIG. 29. Specifically, the judgment in the step S27 results in that the segment_type is an Acquisition Point 
35 DS, but the immediately preceding DS is not stored in the coded data buffer 13. Accordingly, the ignore flag is set to 
0. As a result, the DS10 is loaded to the coded data buffer 13 as shown in FIG. 31 . Suppose, on the other hand, a skip 
operation is made to a picture which is located at a later point than the DS10 (an^ow hst1 in FIG. 30). In this case, the 
DS20 which follows the DS1 0 is loaded to the coded data buffer 13 (arrow hst2 in FIG. 31)). 

[0187] FIG. 33 shows how the DS1 , DS10, and DS20 are loaded when normal reproduction is perfonned as shown 
40 in FIG. 32. Of the three DSs, the DS1 of which segment_type field shows an Epoch Start DS is loaded to the coded 
data buffer 1 3 (step S23). However, for the DS1 0 of which segment type field shows an Acquisition Point DS, the ignore 
. flag is set to "1" (step S29). Consequently, the functional segments of DS1 0 are not loaded to the coded data buffer 
13 but ignored (step S24). As for the DS20, the segment__type field shows an Acquisition Point DS. but the 
command„update_flag is set to "1". Thus, the step 31 results in "YES". Consequently, the button command is loaded 
45 to the coded data Buffer 1 3, thereby replacing the cun-ently stored button command within the ICS of the DS. Yet, the 
ignore flag is set to "1 and thus other data than the button command is not loaded but ignored. 
[0188] With the above operations, by the time the DS20 is presented, the button command LinkPL (#5) associated 
with the same graphkis as DS10 has been replaced by LinkPL (#10). With. this xeplacement, such control is possible 
that the button command associated with the same button changes as the reproduction proceeds. 
50 [0189] Next, a description is given to the processing perfonned by the graphics controller 17. FIG. 34 shows the 
main routine of the processing perfomned by the graphics controller 17. The flowchart shows that the following three 
operations are repeated: time stamp synchronization (step S35); animation presentation (step S36); and UO processing 
(step S37). 

[0190] FIG. 35 is a flowchart of the synchronization control carried out based- on the time stamps. In the flowchart, 
55 judgments are made in the step S43-S47 as to whether specific events have occurred. An occurrence of each event 
calls a respective subroutine which retunes to the main routine after predetermined steps are performed. 
[01 91 ] In the step 843, a judgment is made as to whether the current reproduction point has reached the'time shown 
by the PTS of the CDS. If judged in the affinnative, the rendering to the graphics plane 8 is peri'ormed (step S51), and 
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then the processing goes back to the main routine. 

[0192] !n the step S45, a judgment fe made as to whettier the current reproduction point has reached the PTS of the 
!CS. If judged In the affimiatrve, the graphics piane 8 is started to output the stored content. The output is made to the 
CLUT unit 9 where color conversion is made. After the color conversion, the interactive compotation is overlaid with 

5 the content stored in the video plane 6, thereby an initial interactive display is presented (step S52). Then, a variable 
q is set to the value "1" (step S63), and the processing returns to the main routine. Note that the variable q is a global 
variable (I.e. variable valid throughout a plurality of flowcharts) and its meaning is described later. 
[01 93] In the steps S46 and S47, judgments are made as to whether the current reproduction point has reached the 
time specified by the time infomriation within the ICS. 

10 [0194] In the step S46, the judgment is made as to whether the current reproduction point has reached the time 
shown by the selection_time_outj3ts. If the judgment results in the affimiatlve, the button specified by the 
default_acttvated_button__number is activated, and then the processing goes back to the main routine (step S54). 
[0195] In the step S47, the Judgment is made as to whether the current reproduction point has reached the time 
specified by the compositlon„time_out_pts. If the Judgment results in the affirmative, the processing retunes to the 

?5 main routine after clearing the graphics plane (step S56). This concludes the description of the synchronization based 
on the time stamps. In the synchronization, the steps 51 and 54 call subroutines. The subroutine of the step S51 is 
described below with reference to FIG. 36. 

[0196] FIG. 36 is a flowchart of the rendering to the graphics plane 8. The rendering to the graphics plane 8 needs 
to be completed before the presentation of initial interactive display. In the figure, the ODSx represents the ODS having 
20 the PTScorresponding to the current reproduction point, whfch is shown by the PRS (1 0). In steps S61 -563, judgments 
are made as to whether the ODS is the last one of the ODSs necessary for the initial interactive display. If the ODS is 
the last one, the steps S64-S72 are perfonned. 

[0197] In the step 861 , a judgment is made as to whether the default_selected_button number field has a valid value. 

If it is valid, a judgment is made in the step 863 as to whether ODSx is S-ODSsfirst. If the step 863 results in "NO". 
25 the processing of this flowchart is terminated and returns to the main routine. 

[0198] If the step 861 results in "NO", a judgment is made in the step 362 as to whether the ODSx is S-ODSslast. 

If the step S62 results in "NO", the processing of this flowchart is temriinated and returns to the main routine. 

[0199] Jn the step S64, a judgment is made as to whether the segment_type field within the ICS shows an Epoch 

Start DS. If an Epoch Start DS is shown, the step S66 is perfomned to clear the graphics plane 8, and the steps S66-72 
30 are peri'ormed. The duration taken for clearing the graphics plane 8 is the duration cdl shown in FIGs. 23 and 24. If 

the segment_type field does not show an Epoch Start DS, the step S65 is skipped to the steps S66-72. 

[0200] The steps S66-S72 fomn a loop in which each piece of button Infomnation in the ICS is sequentially processed 

(steps S66 and S67). In the loop, the current piece of button information to be processed is denoted as buttonjnfo 

(p). In the step S68, a judgment is made as to whether the button Jnfo (p) corresponds to a default selected button 
35 specified by the default_selected_button_n umber. 

[0201 J If the buttonjnfo (p) does not corresponding to the default selected button, out of the graphics objects stored 

in the object buffer 15, the one specified in the start_objectJd_normal field of the button Jnfo(p) is designated as the 

graphics object(p)(step S69). 

[0202] On the other hand, if the buttonjnfo (p) con^esponds to the default selected button, out of the graphics objects 
40 stored In the object buffer 1 6 , the one specified by the start_objectJd_selected field of the buttonjnfo (p) is designated 
as the graphics object (p) (step S70). Then, the button (p) is designated as the current button (step S71). The current 
button refers to the one having the selected state in the currently presented interactive display. The reproduction ap- 
paratus stores the ID of the current button in the PRS(IO). 

[0203] Once the graphics object (p) is designated through the steps S69 and S70, the graphics object (p) is rendered 
45 onto the graphics plane 8 at the position shown by the button_horizontal__position and button__vertica(_position fields 
(step S72). The above processing is repeated for each piece of button infonmation within the ICS. In doing so, of a 
plurality of graphics objects associated with each button state, the first graphics object is rendered for each button on 
the graphics plane 8. The duration taken for rendering all the graphics object stored in the object buffer 1 5 is the duration 
td1 shown in FIGs. 23 and 24. This concludes the description of the step S61. Next, with reference to FIG. 37, a 
50 description is given to the subroutine called in the step S54. > . . 

[0204] FIG. 37 is a flowchart of the processing for automatic activation of the default selected button, Rrst of all, a 
judgment is made as to whether the defauit_activaTed_button_n umber field is set to "00'' or "FF" (step S91). If the field 
is set to "00", the processing returns to the main routine without performing any operation. On the other hand, if the 
default_activated_button„number is set to "FF", the current button is designated as the button (u) (step S95). 
55 [0205] If the field value is neither "00" nor ''FF", the button specified by thfe default_activated_button_number field is 
designated as the button (u) (step S92) and activates the button (u) (stepS93) . This button state transition is carried 
out by rendering the sequence of graphics objects from the one specified by the start_object_id_activated field and to 
the one specified by the end_objectJd_activated field in the button Jnfd (u). The rendering position on the graphics 
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plane 8 ts shown by the button_hori2ontal_position and bLitton__vertical_posftion fields. Then, the button command 
associated with the button (u) is executed (step S94), and the processing returns to the main routine. 
[0206] With the above operations, the button in the selected state goes into the activated state at the end of a pre- 
determined duration. This concludes the description of the flowchart shown in FIG. 37. 
5 [0207] Next, a description is given to the animatedpresentation of buttons m the menu (step S36). FIG. 38 Is a 
flowchart of the processing for animated presentation. 

[0208] The initial display has been presented by rendering onto the graphics plane 8, the graphics objects specified 
in the start_object Jd_nonnal and start_objectJd_selected fields of each piece of button Jnfo. To present the buttons 
in animation, for each iteration of the loop of the steps S35-S37, the graphics plane 8 is updated by ovenwriting each 
button with an arbitrary frame (the {q}th graphics object) of the button. To be more specific, the update is made to the 
graphics plane 8 by sequentially rendering the graphics objects specified by the normaLstateJnfo and 
selected_state_info in each piece of button_tnfo. Here, the variable q is used to refer to an individual graphics object 
specified in the normaLstateJnfo and selected_stateJnfo fields in each piece of button Infomiation. 
[0209] Now, a description is given to the processing for making the animated presentation, with reference to FIG. 

^5 38. In the flowchart, the rendering processing is described on the precondition that animation of every button is pre- 
sented with the same number of frames. This is for the sake of simplicity in the description. The presentation of buttons 
with different numbers of frames requires more complex processing. Further, also for the sake of simplicity in the 
description, the repeat_.normal_f1ag and repeat_selected_flag fields are both assumed to be set to the value requiring 
the animation to be continuously repeated. 

20 [021 OJ in a step S80, a judgment is made as to whether the initial display has been presented. If the initial display 
has not yet presented, the processing returns to the main routine without performing and operation. If the initial display 
has been presented, steps S81-S90 are performed- The steps S81-S90 form a loop in which the steps S83-S87 are 
repeated for each piece of button Jnfo included in the ICS (steps S81 and S82). 

[0211] In the step S83, a judgment is made as to whether the button_info (p) corresponds to the current button. 
25 [0212] If the button_info (p) does not correspond to the current button, the variable q Is added to the value of 
start_Object__id„normal in the„button_info (p) to obtain an lD(q) (step S84). 

[0213] If the button Jnfo (p) corresponds to the current button, the variable q is added to the value of 
start_objectJd_selected field in the button Jnfo(p) to obtain an ID(q) (step S85). 

[021 4] Out of the graphics objects stored in the object buffer 1 5, the graphics object (q) is rendered onto the graphics 
30 plane 8 at the position specified by the button_hori2onta!_position and the button„verticaLposltion fields of the 
button Jnfo (p) (step S87). The above operations are repeated for every piece of button Jnfo within the ICS (steps S81 
and S82). 

[0215] Through the above loop, among the graphics objects for presentation of the current button In the selected 
state and other buttons in the normal state, the (q) th graphics objects are rendered on the graphics plane 8 for each 
35 button. 

[0216] in the step S88, a judgment is made as to whether the value of the end__object„ id_normal field has reached 
the value obtained by the start_objectJd_normal field + q (step S89j. If judged in the affirmative, the variable q is 
initialized to "O** and the processing returns to the main routine (step S89). If judged in the negative, the variable q is 
incremented by 1 , and the processing returns to the main routine (step S90). 

40 [0217] Through the steps S80-S90, the presentation of each button on the interactive display is updated using a new 
graphics obj ect each time the loop of steps S35-S37 is executed. By repeating the steps S35-S37 for multiple times, 
each button is presented in animation. When presenting the buttons in animation, the graphics controller 17 makes 
the adjustment so as to present the graphics objects at the rate shown by the animationJrame_rate„code. Such 
adjustment is made also in the other flowcharts when applicable. This concludes the description of processing for 

45 animated presentation. Next, a description is given to the UO processing shown in the step 837 of the main routine, 
with reference to FIG. 39. 

[0218] FIG. 39 is a flowchart of the UO processing. In the flowchart, judgments are made in steps S1 00-S1 03 as to 
whether specific events have occurred. An occurrence of each event calls.a. respective subroutine which returns to the 
main routine after predetemrjined steps are perfonned. In the step SI 00, a judgment is made as to whether the 
50 U0_maskjable field is set to "1 If the filed is set to **1 **, the processing returns to the main routine without perfomiing 
any operation. 

[0219] in the step S101 , the judgment is made as to whether any of therMove Up, Down, Left, and Right keys is 
pushed. At a push of any of the keys, another button is designated as the current button accordingly (step S104), 
followed by the judgment as to whether the auto„action_flag of the newly designated current button is set to "01 " (step 
55 SI 08), If the auto_aGt{onJlag is not set to "01", the processing returns to the main routine. On the other hand, if the 
auto_action Jlag is set to "01 the processing goes onto the step SI 05. ; 

[0220] In the step 31 02, a judgment is made as to whether the Activate key ispushed. At a push of the Activate key, 
the current button is activated in the step S1 05. This button state transition is cam'ed.oul by rendering onto the graphfcs 
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plane 8 the sequence of graphics objects from the one specified by the start_object_id_activated field and the one 
specified by the end„ob]ectJd_activated field of the current button. The rendering position in the graphics plane 8 is 
specified by the button_horizontal_posrtion and the button_vertical_pos!tion fields of the current button. With this ar- 
rangement, such control is realized that the character serving as a button appears to make motion in response to a 
5 user operation. After the button state transition, the button command associated with the current button is executed 
(step SI 06). 

[0221] In a step S1 03, a Judgment Is made as to whether a numeric input is made. If a numeric input is made (step 
S107), operations are performed according to the inputted numeral (step SI 07), and the processing returns to the 
main routine. In FIG. 39, the steps SI 04 and SI 07 each call a subroutine. FIGs. 40 and 41 show the subroutines. Now, 

10 a description is given to those flowcharts. 

(0222] FIG. 40 Is a flowchart of the current button change processing. The flowchart is described on the precondition 
that animation of every button is presented with the same number of frames. This Is for the sal<e of simplicity in the 
description. The presentation of buttons with different numbers of frames requires more complex processing. First, rt 
is detennined which of upper_button_number, lower__button_n umber, left_button_number, and right_button_n umber 

IS in the neighborjnfo of the current button corresponds to the pushed key (step S11 0). 

[0223] Let the button (i) denote the current button and the button (]) denote the button to be designated next as the 
current button (step S111). in a step S1 12, a judgment is made as to whether the button (J) designated in the step S1 11 
is equal to the button (i). If so, the processing returns to the main routine without perfonning any operation. If not, steps 
S113-S120 are perfomned. In the steps S11 3-S1 20, the button (J) is put into the selected state and the button (i) is put 

20 back to the normal state. Each button state is presented in animation. To this end, a variable r is first initialized to "0" , 
The variable r is used to referto an individual frame of the animation. lnthestepS113-S119,the (r)th one of the graphics 
objects associated with the button (i) in the normal state and the (r)th one of the graphics objects associated with the 
button (j) in the selected state is repeatedly rendered onto the graphics plane. 

[0224] Specificaily, the ID(r) is calculated by adding the variable r to the value of the start_object_id_nonTiaI field of 
25 button info (i) (step S114) . After calculating the ID(r) in the above manner, out of the graphics objects stored in the 
object buffer 15, the one having the ID(r) is rendered onto the graphics plane 8 at the position shown in the 
button_horizontaLposition and button_verticaLposition fields of the button Jnfo(i) (step 8115). 

[0225] Next, the ID (r) is then calculated by adding the variable r to the value of the start_objectJd__seIected field of 
the button info (j) (step S116). After obtaining the ID(r) in the above manner, out of the graphics objects stored in the 

30 object buffer 15, the one having the ID(r) is rendered onto the graphics plane 8 at the position shown in the 
button„hori2ontal_posrtion and button_yertical_position fields of the button Jnfo(|) (step S1170. 
[0226] In the step S11 8, a judgment is made as to whether the ID obtained by adding the variable r to the value of 
start_object Jd_normai field is equal to the value of end_objectJd_nonnaI field. If the values are not equal, the variable 
r is incremented (step SI 20), and the processing goes back to the step S114. The steps S114-S1 20 are repeated until 

35 the judgment in the step S118 results in "YES**. By repeating the above steps, the button state transition is achieved 
such that characters serving as buttons make motion in response to a user operation. When the judgment in the step 
S118 results in "YES", the button 0) now designated as the current button (step S119) and the processing returns 
to the main routine. 

[0227] FIG. 41 is a flowchart of the numeric input processing. First, a judgment is made as to whether there is a 
40 piece of button Jnfo (j) having the button_number matching the Inputted numeral (step SI 21). Next, a judgment is 
made as to whether the numerically_selectable_fiag field in the button Jnfo (j) is set to "I " (step SI 22). If both the steps 
8121 and SI 22 result in "YES", a judgment is made as to whether the auto„action„flag field in the button Jnfo (j) is 
set to "01 "(step SI 23). 

[0228] If the auto_action Jlag field value is not "01 **, the sequence of graphics objects from the one specified by the 
45 start_objectJd_seiected field to one specified by the end object_ id_ selected field associated with the button (j) is 
sequentially rendered at the position specified by the button_horizontaLposition and button_vertical_posltion fields 
associated with the button (j) (step 8124). As a result, the button (j) is put into the selected state. Then, the button (j) 
is designated as the current button (step S125), and the processing retums to the main routine. 
[0229] On the other hand, if the auto„action Jiag field value is "01 the current button is activated in the step S126. 
50 This button state transition is carried out by sequentially rendering the .sequence of graphics objects from the one 
specified by the start_object_id_activated field to the one specified by the end_objectJd_actfvated field associated 
with the current button onto the graphics plane S at the position specified by the button_hori2ontal_posltion and 
button_verticalj3osition fields associated with the current button. In a step SI 27rthe button command associated with 
the button (j) is executed, and the processing returns to the main routine. • • - ^ 
55 [0230] If either of the steps S1 21 -SI 23 results in "NO", the processingi returns to the main routine. 

[0231] This concludes the processing performed by the graphics controller 17 for synchronous display. For presen- 
tation of interactive display, such as popup display, triggered by a user operation, the stream graphics processor 14 
and the graphics controller 1 7 perfonm the following operations, which are similar to those perfomned for synchronous 
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display. By doing so, the graphics piane 8 obtains necessary graphics objects. After obtaining the graphics objects, no 
operation is performed until the current reproduction point reaches the time specified by the PTS attached to the ICS. 
If the UO controller 18 receives a user operation of a menu call after the current reproduction point has passed the 
time specified by the PTS, the graphics objects stored in the graphics plane 8 are outputted to the GLUT unit 9 where 
s the outputted graphics objects are overlaid with the.video plane. The output performed in synchronization with the UO 
leads to presentation of the Popup display in response to the menu call 

[0232] As described above, according to the present embodiment, Epochs each composed of an fCS and ODSs are 
multiplexed in an AV Clip. With such an AV Clip, interactive control is readily descnTaed such that the reproduction 
apparatus perfomns a specific operation in synchronization with the presentation of a specific frame of the video. In 

^0 short, the AV Clip of the present invention is suitable for describing interactive control closely bound to the presentation 
tBTJing of video frames. Furthermore, since the Epochs are multiplexed within the AV Clip, even when reproduction of 
hundreds of video segments js requested, it is not necessary to store all the related Epochs to the memory. Epochs 
are sequentially read from the BD-ROM with video packets. Thus, the ICS associated with the video segment currently 
reproduced stays resident in the memory throughout the duration of that video segment. Upon completion of the video 

IS segment, the Epoch having been reproduced is removed from the memory and an ICS corresponding to the next video 
segment is then loaded to the memory. Since the Epochs are multiplexed in the AV Clip, the required size of memory 
is kept to a minimum even if the number of Epochs exceeds several hundreds. 

{SECOND EMBODIMENT) 

20 

[0233] A second embodiment of the present invention relates to the improvement on a click sound reproduced at 
the time when the button state is changed to the selected state and the activated stated. Suppose, for exannple. a user 
interact with the characters appearing in the movie serve as buttons as shown in FIGS. 16 and 17. In such a case, if 
the voice of a respective character is reproduced as a click sound in response to the user operations, the user can 

25 instinctively recognize which of the buttons the user is operating. With this arrangement, it is easier for users to rec- 
ognize the button operations. Here, a problem arises as to how to simultaneously reproduce the click sound and the 
main sound. The main sound used herein refers to the sound of the movie, such as the speech of the characters and 
background music in the movie. The audio stream carrying the main sound is multiplexed into the AV Clip with the 
video and graphics streams. The audio decoder 7 decodes the audio stream. At the time of reproducing the click sound, 

30 the main audio needs to be muted. To this end, the operations of audio decoder 7 need to be suspended. However, 
the suspension of audio decoder 7 Involves the risk of producing noise. 

[0234] In order to avoid the production of noise due to the suspension of audio decoder 7, the reproduction apparatus 
has the internal structure as shown in FIG. 42. The structure shown in this figure is similar to the one shown in the FIG. 
25 with the addition of a preload memory 21 and a mixer 22. 

35 [0235] TTje preload memory 21 stores In advance uncompressed LP CM data to be reproduced as s click sound. 

[0236] The mixer 22 mixes the uncompressed LPCM data stored in the preload memory 21 with the output of the 
audio decoder 7. The data is mixed at the rate instructed by the graphics controller 1 7 included In the graphkis decoder 
12 (see FIG- 26). Since a click sound is reproduced in accordance with the mixing parameters, it is not necessary to 
suspend the output of audio decoder 7. 

40 [0237] This concludes the description of the reproduction apparatus structure according to the second embodiment. 
[0238] in order to simultaneously reproduce the main sound and the circk sound, the decompressed LPCM data 
recorded on the BD-ROM needs to be loaded to the preload memory 21 in advance. Unfortunately, however, the size 
of decompressed LPCM data Is relatively large. For example, less than ten seconds of 1 6-bitMB kHz LPCM audio data 
has the size of one megabyte. 

45 [0230] In order to meet the demands for reducing the size of preload memory 21 , the ICS according to the present 
invention has the data structure as shown in FIG. 43. FIG 43 shows the data structure of ICS for reproducing the click 
sound in the above manner. The data structure in the figure differs from the one shown in FIG. 1 1 in the button_info. 
In each piece of button info, the selected__state_info() and the activated_state_info() each additionally have 
"audio_speciftcationJnf o" field and **audio_reproduction_controUnfo" field. 

50 [0240] The audio_specification_info field shows a file name or ID of audio data to be read and reproduced as a click 
sound by the reproduction apparatus when the button associated with the buttonjnfo is put into a different state. The 
dfck sound data to be loaded to the preload memory 21 is .specified -by the seiected_stateJnfoO and 
activated„state_info {) of the button Jnfo. The click sound data loaded tothe preload memory is supplied to the mixer 22. 
£0241 } The audio_reproduction_controLinfo is composed of a plurality of mixing parameters. Each mixing parameter 

55 shows the rate at which each audio component is to be mixed with the main sound. Each mixing parameters takes on 
a value within a range of 0-1 .0. The click sound data is multiplied by the values shown by the mixing parameters before 
being reproduced. In the case where the audio data is composed of R and L components, the 
audio„reproduction_controUnfo field shows separate mixing parameters' for R and L components, so that the mixer 
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22 is instructed to mix the audio data at the mixing rates shown by those parameters. 

[0242] With the provision of the audio_reproduction_controljnfo. the L component of the uncompressed LPCM data 
may represent the dick sound of button A, while the R component representing the dick sound of button B, In this way, 
two different click sounds for two different buttons may be collectively stored as one piece of uncompressed LPCM data, 
5 [0243] In addition to the collectively stored click sounds as described above, the button_info (1) includes the 
audio_reproduction„controLinfo set for output of the R component only, whereas the buttonJnfo{2) includes the 
audio„reproductfon_controUnfo set for output of the L components only. With this arrangement, simultaneously with 
the state transition of the button A to the selected state, the L component of uncompressed LPCM data Is reproduced 
according to the audio reproduction_controUnfo of the button Jnfo (1), so that the cfck sound of button A is repro- 
of duced. 

[0244] Similarly, simultaneously with the state transition of the button B to the selected state, the R component of 
uncompressed LPCM data is reproduced according to the audio_reproduction„controUnfo of the button_info (2), so 
that the click sound of button B is reproduced. 

[0245} With reference to FIGs. 44 and 45, a description is given to a specific example of the click sound reproduction 
1^ using the above ICS and reproduction apparatus. The example related to the state control information shown In FIGs, 
44A and 448. The state control infonnation shown in FIG. 44A includes button„!nfo (1) and button_!nfo(2). As shown 
by the arrows sy1 and sy2 in the figure, both the button_lnfo (1) and button Jnfo 2) specify the same piece of click 
sound data, which is stereo sound. The audio_reproduction_controlJnfo of the button Jnfo (1) includes the mixing 
parameter for L component, whereas the audio_reproduction_controljnfo of the button Jnfo (2) includes the mixing 
^0 parameter for R component, 

[0246] FIG. 44B shows the reading process of the ICS including the state control information. Prior to the ICS, click 
sound data is loaded to the preload memory 21 . 

[0247] FIGs. 45A and 46B illustrate the reproduction control of the click sound data according to the ICS read to the 
preload memory 21 . In the case where the button A associated with button_info (1 ) is in the selected state, the graphics 

25 decoder 12 controls the audio decoder 7 so that the dick sound data is reproduced according to the 
audio_„reproduction„controUnfo of the button Jnfo (1). As a result, the L component of the stereo click sound data is 
reproduced. On the other hand, in the case where the button B associated with button Jnfo(2) Is in the selected state, 
the graphics decoder 12 controls the audio decoder 7 so that the dick sound data is reproduced according to the 
audio_reproduction_control info of the button Jnfo (2). As a result, the R component of the stereo dick sound data 

30 is reproduced. 

[0248] With the above control, a piece of click sound data composed in stereo is used as the dick sound of button 
A by reproducing the L component and as the dick sound of button B by reproducing the R component, when a re- 
spective button is selected, 

[0249J This concludes the specific example in which a plurality of click sounds is collectively stored as one piece of 

55 uncompressed LPCM data. Alternatively, the button„info of the present invention may be set so as to produce the click 
sound upon a button operation in a manner that the sound comes from a different direction. A specific example of such 
control is described below with reference to FIGs. 45. FIG. 45C shows an example of three pieces of buttonjnfo 
associated with three laterallyalignedbuttons (button A, button B, and button C). The buttonjnfo associated with the 
button A, whteh is on the left, has the mixing parameter of 1 .0 for the L component The buttonjnfo associated with 

40 the button B, whrch is in the middle, has the mixing parameters of 0.6 for both the L and R components. The button 
Info associated with the button C, which is on the right, has the mixing parameter of 1 .0 for the R component. With the 
above mixing parameters, when the left button A is selected, the dick sound is outputted from the left speaker When 
the right button C is selected, the click sound is outputted from the right speaker. When the middle button B Is selected, 
the click sound is outputted equally from both speakers. As above, the buttonjnfo may be set so that the click sound 

45 is outputted from a different direction depending on the position of button on the screen. The click sound coming from 
the direction corresponding to the position of the button pushed adds to the realism in the button operations. 
[0250] As described above, according to the present embodiment, a click sound for each of a plurality of buttons is 
integrated into one piece of stereo click sound data. With the use of audio_specificationJnfo and 
audio_reproduct!on„controlJnfo, the same piece of click sound data Is repn^duced as different clfck sounds for different 

50 buttons. With this integration, the amount of dick sound data is reduced, and thus ithe size of preload memory 21 for 
loading the dick sound data can be reduced. r •. - 

[0251] Although the specific examples given above are based on the stereo dfek sound data, the dick sound data 
may be uncompressed 5.2-channel audio data. FIG, 44C shows an example similar to the one shown in FIG. 45C. 
Yet, in the example in FIG. 44C, the mixing parameters are set for reproduction of 5.2-channel audio data. Thus, in 
55 addition to the L and R components, the 5. 2-channel audio data includes Center, Rear Left, and Rear Right compo- 
nents. The buttons A, B, and C are diagonally aligned on the interactive display, in this example, the buttonjnfo as- 
sociated with the button A has the mixing parameter of 1 .0 for L component. The buttonjnfo associated with the button 
C has the mixing parameter of 1 .0 for Rear Right component The buttonlinfo associated with the button B has the 
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mixing parameters of 0.1 , 0.1 , 0.4, 0.2, and 0.2 for L, R, Center, Rear Left. Rear Right components, respectively. With 
this setting, when the button A is selected, the clicic sound is outputted from the right. When the button B is selected, 
the click sound is outputted from the left. When the middle button B is selected, the click sound is outputted from alt 
the directions. As above, the direction from which the click sound comes is changed depending on the position of the 
button pushed. This arrangement adds to the realism in button operations (Note in the above example, the button_info 
associated with the button B may alternativefy be set to have the parameter of 1 .0 for the Center component and the 
parameter of 1 .0 for each of the other components). 

[0252] Furthermore, in addition to the audio decoder 7, another audio decoder may be provided for click sound data. 
In this case, the preload memory 21 stores compressed audio data in advance. In response to the button state transition, 
the audio decoder for click sound data extracts the compressed audio data from the preload memory 21 and decodes 
the extracted data. The provision of the audio decoder for dick sound data allows the preload memory 21 to store the 
compressed audio data, so that the size of preload memory 21 can be reduced. 



(THIRD EMBODIMEhsTT) 

15 

[0253] A third embodiment of the present invention relates to a manufacturing process of the BD-ROM. FIG. 46 is a 
flowchart of the manufacturing process of the BD~ROM according to the present invention. 

[0254] The manufacturing process includes a material production step of recording video, sound, and the like (step 
S201 ), an authoring step of creating an application forniat using an authoring device (step S202), and a pressing step 
20 of creating an original master of the BD-ROM and perfonning stamping and bonding to complete the BD-ROM (step 
S203) . 

[0255] The BD-ROM authoring step includes the following steps S204~S209. 

[0256] First, in the step S240, videomateriais, audiomaterials, and supplemental graphics materials are encoded 
into a video stream, an audio stream, and a graphics stream, respectively. Next, in the step S205, the verification test 
25 of the graphics stream is carried out. As described in the first embodiment, the graphics stream includes, in addition 
to graphics data for presentation of buttons, state control Infomiation of the button. This makes it possible to verify the 
graphics stream independently. If any error is detected (step S206: NO), an appropriate con-ection is made to the 
graphics stream, which is not yet multiplexed with the other streams (step S207), and the verification test of the graphics 
streams is carried out again. 

30 [0257] If no error is detected through the verification test of graphics stream (step S206 : YES), the video, audio, 
graphics streams, which are obtained in the step S208 by encoding the materials, are interieaved multiplex converted 
to one digital stream. In the step S209 that follows, necessary information is generated based on the scenario for the 
BD-ROM, and the scenario and digital stream are adapted to be in compliance vWth the BD-ROM format. 
[0258] According to the present embodiment, the ICS defining the button state transition is integrated into the graphics 

^ stream with graphics data. Thus, it is no longer necessary to wait forthe video stream to be encoded or the multiplexing 
of streams to be completed. Instead, the graphic stream is ready for verification test upon Its production to see how 
the button states change as the reproduction proceeds. Since the verification test of button state transition can be 
carried out at an eariier stage of authoring, a possibility is avoided that an error is detected right before the shipment, 
which forces the developers to rush. The verification test of graphic stream alone provides a better environment for 

40 incorporating animated buttons of complex motion into a movie. 



(SUPPLEMENTAL NOTE) 



[0259] Practicing of the present invention is not limited to the specific examples described above. The present in- 
vention can be practiced with any of modifications (A) to (O) below. The invention of each of the claims of this application 
broadened or generalized descriptions of the above-described embodiments and their modifications including the fol- 
lowing. The extent of the broadening and generalization reflects the state of ttie art In the related technical field at the 
time of filing of the present application. Yet, the invention recited In eachrciaim Is directed to means of solving the 
problems associated with the present invention. Thus, the scope of each Invention does not go beyond the scope in 
which those skilled in the art recognize the means for solving the problems. Consequently, the invention recited in each 
of the appended claims is in substantial correspondence with the detailed description above. 



(A) The above embodiments describe the case where the BD-ROM is used as the recording medium. Main features 
of the present invention, however, lie in a graphics stream recorded on the recording medium, which does not rely 
on physical characteristics of BD-ROMs. Therefore, the present invention is applicable to any recording medium 
that is capable of recording a dynamic scenario and a graphics streann. Examples of such a recording medium 
include: an optical disc such as a DVD-ROM, a DVD-RAfvl, a DVD-RW, a DVD-R, a DVD+RW, a DVD+R, a CD-R, 
or a CD-RW; a magneto-optical disk such as a PD or an MO: a semfconductor memory card such as a CompactFlash 
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card, a SmartMedia card, a Memoty Stick card, aMultlMedtaCard, or a PCMCIA card; a magnetic disk such as a 
flexible disk, SuperDisk, Zip, or Cliki; a removable hard disk drive such as ORB, Jaz, SparQ, SyJet, EZFley, or 
Microdrive; and a non-removable hard disk drive. 



5 (B) The above embodiments describe the cases where the reproduction apparatus decodes an AV Cilp on the 

BD-ROM and outputs the decoded AV Clip to the television. As an alternative, the reproduction apparatus may be 
equipped with only a BD drive, with the remaining construction elements being provided in the television, in this 
case, the reproduction apparatus and the television can be incorporated in a home networi< connected with an 
IEEE 1394 connector. Furt:hemnore, the above embodiments describe the cases where the reproduction apparatus 

'fo is connected to the television, but the reproduction apparatus may instead be integrated with a display device. 

Also, the reproduction apparatus may include only the elements relating to the essential part of processing. Those 
reproduction apparatuses are all descried in this speciTication as inventions. Accordingly, regardless of the modes 
therefor, an act of manufacturing a reproduction apparatus based on the internal structure of the reproduction 
apparatus described in the first, second, or third embodiment Is considered to be an act of practicing the present 

15 invention. Also, any act of assigning with charge (i.e. for sale) or without charge (i.e. as a gift), leasing, and importing 

the reproduction apparatus is an act of practicing the present invention. Likewise, an act of offering for assignment 
or lease of the reproduction apparatus using storefront displays, catalogs, or brochures is an act of practicing the 
present invention. 



20 (C) infomiation processing using the programs shown in the flowcharts Is actually realized iislng hardware re- 

sources. Accordingly, the programs which describe the operational procedures shown in the flowcharts are them- 
selves an invention. The above embodiments describe the cases where the programs are incorporated in the 
reproduction apparatus, but the programs can be used independently of the reproduction apparatus. Acts of prac- 
ticing the programs include (1) an act of manufacturing, (2) an act of assigning with or without charge, (3) an act 

2S of leasing, (4) an act of importing, (5) an act of providing to the public via a bi-directional electronic communications 

network, and (6) an act of offering for assignment or lease using storefront displays, catalogs, or brochures. 

(D) The time elements of the steps which are executed in a time series in each of the flowcharts can be regarded 
as the necessary elements of the present invention. With this being so, the procedures shown by these flowcharts 
30 are considered to disclose reproduction methods. If the processing shown in each flowchart is carried out by 

performing the steps in a time series so as to achieve the intended aim and the intended effect, this is an act of 
practicing the recording method of the present invention. 



(E) When recording an AV Clip on the BD-ROM, an extension header may be added to each TS packet in the AV 
35 Clip. The extension header is called a TP_extra_header, includes an arrivaLtime__stamp and a 

copy_43ermission„indicator, and has a data length of 4 bytes. TS packets with TP_extra__headers (hereafter "EX 
TS packets") are grouped in units of 32 packets, and each group is written to three sectors. One group made up 
of 32 EX TS packets has 6,144 bytes (=32 x 192), which is equivalent to a size of three sectors that is 6, 144 
bytes (=2,048 X 3). The 32 EX TS packets contained in the three sectors are called an Aligned Unit. 

40 In a home networic connected with an IEEE 1394 connector, the reproduction apparatus 200 transmits an 

Aiigned Unit in the following manner. The reproduction apparatus removes a TP_€xtra_header f rom each of the 
32 EXTS packets in the Aligned Unit, encrypts the body of each TS packet according to the DTCP Specification, 
and outputs the encrypted TS packets. When outputting the TS packets, the reproduction apparatus inserts an 
isochronous packet between adjacent TS packets. A point where the isochronous packet is inserted is determined 
based on a time shown by an arrlval_time_stamp of the TP„extra_header. The reproduction apparatus 200 outputs 
a DTCP„descriptor, as well as the TS packets. The DTCP__descriptor corresponds to acopy__permissionJndicator 
in the TP_extra_header- With the provision of the DTCP_descriptor indicating "copy prohibited", It is possible to 
prevent, when using the TS packets in the home network connected with the IEEE 1 394 connector, the TS packets 
from being recorded to other devices. 

50 i:: 

(F) The above embodiments describe the cases where an AV Clip of the BD-ROM Fomiat is used as a digital 
stream, but the present invention can also be realized with a VOB (Video Object) of the DVD-Video Format or the 
DVD-Video Recording Format. The VOB is a program stream that coniplies with the ISO/IEC 13818-1 Standard 
and is obtained by multiplexing a video stream and an audio stream. Also, the video stream in the AV Clip may be 

55 an MPEG4 video stream or a WMV video stream. Further, the audio stream in the AV Clip may be a Linear PCM 

audio stream, a Dolby AC-3 audio stream, an MPS audio stream, or en MPEG-AAC audio stream. 

. . ■ . , 

(G) The video editing described in the above embodiments may be obtained by encoding an analog image signal 
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broadcast by analog broadcasting. 

Alternatively, analog/digital image signals recorded on a videotape may be encoded to obtain contents. Also, 
analog/digital image signals directly captured by a video camera may be encoded to obtain contents. A digital work 
distributed by a distribution server is applicable too. 

5 

(H) The ICS may define the operations to be perlomied at the time of timeout. The timeout of the ICS is specified 
by the composition_time_out_pts described in the first embodiment. FIG. 47 shows an ICS according to one mod- 
ification of the present invention. The ICS in the figure is newly provided with a Still/Pause Jnfomnatton field. The 
StilUPauseJnformation field shows whether the operations of reproduction apparatus are to be brought into "Stiir* 

10 or "Pause". The operations of the reproduction apparatus referred herein include the decoding operations by the 

video decoders, stream graphics processor 14, and graphics decoder 12 as well as the navigation operations by 
the graphics controller 1 7 and the controller 20. The tenm "Still" used herein refers to suspension of both the decode 
operations and the navigation operations, whereas the term "Pause" refers to suspension of the decode operations 
while the navigation operations are continued. Under the Stil! condition, the navigation operations are suspended, 

15 so that the last reproduced picture remains displayed as a still picture and no button state transition is possible. 

On the other hand, under the Pause condition, the navigation operations are continued, so that users are 
allowed to change the button states. With provision of the Pause/Stilljnformation in the ICS, control to be executed 
upon timeout can be defined at the time of authoring. 

20 (I) In the second embodiment, the buttonjnfo defines a click sound for each button. In addition, a click sound may 

be defined by ICS for each key of the remote controller. FIG. 48 is a view showing the ICS defining a click sound 
for each key of the remote controller 

The upper_audio field defines the audio specification infomiation and reproductton control infomnation to be 
referred to at a push of the Move Up key. 
25 The {ower„audto field defines the audio specification infomnation and reproduction control infomiation to be 

referred to at a push of the Move Down key. 

The left_audio field defines the audio specification infomnation and reproduction control infomiation to be re- 
ferred to at a push of the Move Left key. 

The right_audio field defines the audio specifbation infomnation and reproduction control information to be 
30 refenred to at a push of the Move Right key. 

The activated_audlo field defines the audio specification information and reproduction control to be referred 
to at a push of the Activated key. At a push of a key of the remote controller 400, the preload memory 21 and the 
mbcer 22 operate according to the audio specification information and reproduction control information that are 
associated with the pushed key, so that a corresponding click sound is reproduced. 

35 

(J) Graphics objects described in the above embodiments is run-length encoded raster data. Run-length encoding 
is used for compression/encoding of graphics objects, because the run-length encoding is suitable for compression 
and decompression of subtitles . Subtitles have a property in that a continuous length of the same pixel value in 
a horizontal direction is relatively long. Therefore, by perfonning compression using run-length encoding, a high 
40 compression rate can be attained. In addition, ain-length encoding reduces a load for decompression, and is 

therefore suitable for realizing decoding by software. Nevertheless, the use of run-length encoding for graphk» 
objects is not essential to the present invention. For example, graphfcs objects may bePNGdata. Also, graphics 
objects may be vector data instead of raster data. Further, graphics objects may be transparent pattems. 

45 INDUSTRIAL APPLICABILITY 



[0260] A recording medium and a reproduction apparatus according to the present invention realize interactive control 
on a movie, thereby adding value to the movie. Movies distributed with such added value invigorate the movie market 
and consumer appliance market. Thus, the recording medium and the reproduction apparatus according to the present 
50 invention are highly applicable in the movie and the consumer appliance industries. 



Claims 7;:. 
55 1. A recording medium comprising: ■ 

a digital stream into-which a video stream and a graphics stream are multiplexed; 
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wherein said graphics stream includes graphics data and state control infonnation integrated therein, the 
graphics data being used to compose an interactive display, and the state control infonir>ation being used to cause 
the interactive display to change to a different state in response to a reproduction proceeding of the video stream 
arrd a user operation. 

2. The recording medium according to Claim 1 , wherein: 

said graphics stream is a packet string; 

the packet string includes at least one set of packets, each set of packets includes a packet containing state 
control infomnation and a packet containing graphics data; and 

the packet containing state control infonnation is attached with a time stamp indicating a display timing of a 
picture to be displayed in synchronization with the interactive display. 

3. The recording medium according to Claim 1 , wherein: 

said graphics stream includes a plurality of display sets, each of the plurality of display sets including state 

control information and graphk^s data; 

the state control information includes type information; and 

the type information is used to tndk^te whether a display set to which the state control infonnation belongs is 
identical in contents to an immediately preceding display set on said graphk^s stream. 

4. The recording medium according to Claim 3, wherein: 

the state control infomnation in each of the plurality of display sets Includes an update flag; 
the update flag, when set to ON, indicates that the display set is identical to an immediately preceding display 
set with respect to the state control information and the graphics data, except for a button command; 
the update flag, when set to OFF, indicates that the display set is identical to an immediately preceding display 
set with respect to the state control information and the graphics data; and 

the button command is for execution by a reproduction apparatus upon activation of an associated button on 
the interactive display. 

5. The recording medium according to Claim 1 . wherein: 

the interactive display presents n buttons, and an arbitrary button / of the n buttons has m states, wherein /, 

n, and m are integers; 

the state control information includes n pieces of button infonnation, and a piece of button infonnation asso- 
ciated with the button / includes m pieces of state infonnation; and 

when an arbitrary one of the m states of the button / is a state /, a fih piece of state infonnation indicates whbh 
piece of the graphics data is to be used for presentation of the button / in the state / wherein / is an Integer 

6. The recording medium according to Claim 6. wherein: 

the button information / includes neighbor information; and 

the neighbor infonnation indbates one of the n buttons to receive a currently selected state in response to a 
user operation specifying a direction when the button / is in the currently selected state, 

7. The recording medium according to Claim 5. wherein: ' 



each of the n buttons has a number assigned thereto; and ; 

each of the n pieces of button infonnation includes a number assigned to an associated button and a flag 
showing whether the button is numerically selectable with the number. 

8. The recording medium according to Claim 5, wherein: 

the button information / includes click sound controi infomnation; and 

the click sound control information indicates a piece of audio data to be reproduced upon a state transition of 
the button /, and indk:ates a control to be executed by a reproduction apparatus for the audio data reproduction. 
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9. The recording medium according to Claim 1 . wherein: 
said graphics stream is a packet string; 

the pacl<et string includes at least one set of packets, each set of packets includes a packet containing state 

5 control Infonr^ation and a packet containing graphics data; 

the packet containing state control infomnation is attached with a time stamp; and 

the time stamp indicates, on a reproduction time axis of the video stream, a time after which the interactive 
display is available for presentation in response to a user operation. 

^0 10. A reproduction apparatus for reproduang a digital stream into which a video stream and a graphics stream are 
multiplexed, said reproduction apparatus comprising: 

a video decoder operable to decode the video stream to obtain video data; and 

a graphics decoder operable to decode the graphics stream to obtain an interac^ve display; wherein: 

15 

the graphics stream includes graphics data and state control infonnation; 

the state control information being used to cause the interactive display to change to a different state in 
response to a reproduction proceeding of the video data and a user operation; and 
said graphics decoder including 

20 

a processing unit operable to decode the graphics data Included in the graphic stream to obtain the 
Interactive d^play, and 

a controller operable to control a state of the interactive display based on the state control information. 
25 11 , The reproduction apparatus according to Claim 1 0, wherein: 

the graphics stream is a packet string; 

the packet string includes at least one set of packets, each set of packets includes a packet containing state 
control information and a packet containing graphics data; 
30 the packet containing state control information Is attached with a time stamp indicating a display timing of a 

picture to be displayed in synchronization with the interactive display; and 

the decoding by said processing unit and the control by said controller are performed with reference to the 
time stamp attached to the packet containing state control information. 

35 12. The reproduction apparatus according to Claim 10, wherein: 

the graphics stream includes a plurality of display sets, each of the plurality of display sets Including a packet 
containing graphics data and a packet containing control information; 
the state control information includes type information indicating a type of the display set: 
40 when performing nonnal reproduction, said controller ignores a display set of whteh type infonnation indicates 

that the display set is identical in contents to an immediately preceding display set; and 
when performing reproduction starting from a point searched in a skip operation, said controller decodes a 
display set that is located after the reproduction start point and that has type information indicating that the 
display set is identical in contents to an immediately preceding display set. 

45 

13. The reproduction apparatus according to Claim 12, wherein: 

the state control information in each of the plurality of display sets includes an update flag; and 
when perfomiing nonnal reproduction, said controller reads a button command if the update flag Is set to ON 
so and the type infomnation indicates that the display set is identfcai: in contents to an immediately preceding 

display set. 

14. The reproduction apparatus according to Claim 10, wherein: 

55 the interactive display presents n buttons and an arbitrary button /of the n buttons has m states, wherein /, n, 

and m are integers; 

the state control information Includes n pieces of button information, and a piece of button infonriation asso- 
ciated with the button / includes m pieces of state infomnation; and 
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when an arbitrary one of the m states of the button / is a state /, said controller presents the button / in the 
state / using a piece of graphics data specified by a jth piece out of the m pieces of state information, wherein 
/is an integer. 

15. The reproduction apparatus according to Claim 14, wherein: 

the button infomnation / includes neighbor information; and 

when the button / is in a currently selected state and a user operation is made to transfer the currently selected 
state, said controller puts the button / back into a normal state and puts a button specified by the neighbor 
information out of the n buttons into the currently selected state. 

16. The reproduction apparatus according to Claim 10. wherein when a user operation of numerical input is made, 
said controller changes a state of a button to which a value equal to the numerical input is assigned. 

17. The reproduction apparatus according to Claim 16, further comprising: 

an audio decoder operable to decode an elementary stream multiplexed in the digital stream for producing an 
audio output; 

a preload memory operable to store uncompressed audio data; and 

a mixer operable to, upon a user operation to activate a button, mix the uncompressed audio data stored in 
said preload memory with an output of an audio encoder for prpdudng an audio output. 

18. The reproduction apparatus according to Claim 17, wherein: 

the state control information includes a plurality of pieces of button information; 
each of the plurality of pieces of button information includes click sound control information; 
the click sound control information indicates a piece of audio data to be reproduced upon a button state tran- 
sition, and indicates a control to be executed by said reproduction apparatus for the audio data reproduction; 
and 

when a user operation Is made to activate a fdh button on the interactive display, the mixer perfonns the mixing 
in accordance with click sound control information Included in a Mh piece of button information, wherein k is 
an integer. 

19. The reproduction apparatus according to Claim 10, wherein: 

the graphics stream is a packet string; 

the packet string includes at least one set of packets, each set of packets includes a packet containing state 

control information and a packet containing graphics data; 

the packet containing the state control information is attached with a time stamp; 

the time stamp indicates, on a reproduction time axis of the video stream, a time after which the interactive 
display is available for presentation in response to a user operation; 

said reproduction apparatus further comprises a receiving unit operable to receh/e a user operation; and 
said controller executes control to present the interactive display when said receiving unit receives a user 
operation after a cun^ent reproduction point reaches the time shown by the time stamp attached to the packet 
containing state control infomiation. 

20. A method for recording data onto a recording medium, said method comprising: 

generating application data; and : 
recording the application data onto the recording medium, whereinr-i - - : : 

the applfcation data indudes a digital stream into whfch a video stream and a graphics stream are multi- 
plexed; and 

the graphics stream includes graphics data and state control information integrated therein, the graphics 
data being used to compose an interactive display and the state controj infonnation being used to cause 
the interactive display to change to a different state in response to a reproduction proceeding of the video 
stream and a user operation. 
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21. A program for enabling a computer to reproduce a digital stream into which a video stream and graphics stream 
are multiplexed, said program comprising code operable to cause the computer to perfomn: 

decoding of the video stream to obtain video data; and 
s decoding of the graphics stream to obtain an interactive display, wherein: 

the graphics stream includes graphics data and state control information; 

the state control information is used to cause the interactive display to change to a different state in re- 
sponse to a reproduction proceeding of the video data and a user operation; and 
said decoding of the graphics stream includes 

decoding the graphics data included in the graphics stream to obtain the interactive display, and 
controlling a state of the interactive display in accordance with the state control information, 

^5 22. A method for reproducing a digital stream into which a video stremn and a graphics stream are multiplexed, said 
method comprising: 

decoding the video stream to obtain video data; and 

decoding the graphics stream to obtain an interactive display, wherein: 

20 

the graphics stream includes graphics data and state control infomriation; 

the state control information is used to cause the Interactive display to change to a different state in re- 
sponse to a reproduction proceeding of the video data and a user operation; and 

said decoding the graphics data includes 

25 

decoding the graphics data included in the graphics stream to obtain the interactive display, and 
controlling a state of the interactive display in accordance with the state control information. 
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